Python is not the best at anything, but is the second best at almost everything. That’s the reason I use it. Everything I need to do, I know I can do it in python, maybe isn’t going to be the better or faster it could be, but it will work.
This. As a chemist I just need to do rather simple calculations, plots and sometimes use a package or two. My total yearly compute time is like 1 hour in python.
Spending 5 years learning c, assembly, data science and what not just to reduce that to 1 minute is not really worth it.
Having worked on large C++ projects, the solution to this issue is using a good IDE with an LSP. But it is quite a regrettable situation because setting up an LSP can be a difficult, and they also tend to break quite easily. Understanding how to make an LSP work is an important productivity skill in those kinds of positions.
Interesting, I had never heard of ccache before, though yes, all good build systems (CMake, Ninja, etc.) should cache intermediate object files.
But the projects I was working on were so large that even binary and unit test executables were so large that even they would take ~20 seconds to link. You can’t use caching to alleviate that buildtime cost unfortunately.
Python is largely compiled. All the pieces of a compiler are built into how it processes things. Almost nothing works on an interpreter model anymore, where each line is parsed and executed before handling the next. Unix-style shell scripts are one of the very few exceptions. I believe JavaScript also starts being interpreted in the browser in order to start executing immediately, but then a compiled version is swapped into the runtime as soon as it’s ready.
Depends on what you mean by compiled. Python typically translates to meta code and that is interpreted. True compilation to binary runs about 100x faster. Python is slow though there are faster versions and ways around it.
I mean that there are successive steps to transform the entire code into tokens, the tokens into an AST, and the AST into some intermediary or final form.
True compilation to binary runs about 100x faster.
No, it doesn’t. Take a look at any of the number of projects that have attempted to compile Java to native code over the years. You’d be lucky to see any substantive gain at all. They sometimes have a use for packaging everything up in a single distributed binary, but you don’t do it for speed.
Things like C and Rust are fast because the language semantics can be compiled in a fast way.
We will have to disagree on that. This is all problem spectific, but I have found C code integrated via ctypes, cffi, or by a C extension is over 100x Python alone. Interestingly Python, Numba, and Numpy together which is a more pythonic solution can get to those speeds too.
All of the other approaches I have tried are much slower: Nuitka, Cython, Numpy alone, PyPy, etc.
To get best speeds one has to compile for your specific architecture and enable things like vectorization, auto parallel, and fast math. Most default builds including libraries do not do that.
This is all problem spectific, but I have found C code integrated via ctypes, cffi, or by a C extension is over 100x Python alone. Interestingly Python, Numba, and Numpy together which is a more pythonic solution can get to those speeds too.
Of course you did. Those are changing the semantics of the language. For example, things like Numpy store arrays more like how C does it than Python. That makes all the difference, not merely compiling to native code.
You can get about 10x by compiling Python using PyPy. So compiling is not nothing. Using Numpy alone is about 5x which surprised me. There is a lot of missleading stuff out there about how to make Python fast. Lot of people say CPython is pretty fast or that using a binary library like numpy is fast. No CPython is very slow and libraries are not always that fast.
Edit: Another compiler is Numba which is more specialized. It can get 30x on some code without numpy. Again compiling can help.
Yeah, you can use it both for full applications (web or desktop) as well as simple scripts. The flow of getting from something simple to a full blown application is pretty smooth and natural - better than just starting out in Java or C++ and taking at least an hour before you get anything that runs.
Python is popular because it’s whitespace based syntax make noobs think it’s easy to read. This and it’s pre-installation on Linux made it popular amongst academics who embraced it for data analysis. This lead to a lot of data scientists embracing it and writing libraries for it which created a virtuous cycle in that arena.
And it’s a damn shame because Python, and it’s whitespace nonsense and lack of type system, is a horribly impractical language for building anything at scale.
White space nonsense. Yes and no. I do not enjoy tracking down white space issues until I remember the alternative. Matching brackets, writing begin/end statements, and worst of all looking at code that appears well indented until later you find it is not. and it runs totally differently then it looks. Similar issues with semicolons.
Pretty much everyone uses LSEs these days so frankly matters less either way most of the time.
It does make it easier to read. We primarily use indentation to match brackets, so clearly, indentation is the most important readability feature between the two, not brackets.
Try reading the same piece of code: without brackets, and then without whitespaces and tell me which is better.
Whitespaces as syntax force you to indent your code properly and not be misleading.
Brackets are infinitely easier because any IDE (and most text editors) have a dead simple linting plugin which auto indents for you and lets you know when you fucked up. Your editor can barely do anything with whitespace only because everything is valid syntax.
Also lemme take this opportunity to say fuck yaml for exactly this reason.
I literally do that professionally, and I’m telling you that brackets are inarguably better.
You are used to whitespace; I have literally spent months refactoring and upgrading a react frontend and a python backend and am telling you that brackets are a single character and far more predictable, and then your IDE / Linter / Formatter can reliably use them to fill in the perfect indentation and whitespace.
My first professional language was Python, but I have since tried a half dozen others and would never choose to go back to a whitespace based language unless there was a good reason.
I’ve written C, JS, PHP, and a few others. Today I write Python almost everyday and wouldn’t trade it for any of them. Maybe Rust, but not because of brackets.
Typing thing is way over blown. What did suck before Python 3 was indeterminate type result from the / operator. You never know if it would be integer or floating division looking at the code. Now it is floating.
Not only data science libraries, there is a library for about anything. Which is not the case for your list of languages.
Java has an excellent ecosystem for libraries. And a much more mature one in most cases than Python. The Spring framework alone is simply amazing.
Go’s ecosystem is pretty good as well and getting better.
When you understand how python works, it’s quite simple to use too, and has lot less clutter than say java or c#.
The problem is that many python devs consider things like “type hints” and “classes” to be “clutter”. It takes a lot of discipline to write managable python. Being “newb friendly” is a curse. “stringly typed” crap does NOT scale well.
It can scale though. It parallelizes really well if you use queuing systems to distribute the load. You just have to make sure that the hot loops are in C/C++, and it is very easy to interface with compiled binaries via the C API.
Agreed. I have seen a lot of Python code that was really painful to massage back into a more structured object hierarchy. Java certainly does a bit better in that respect, and as a language, it does a much better job of encouraging better practices, but I think it’s also largely due to the kinds of people that use those languages as well.
My opinion: Python may not be the best at everything it does, but it’s in the top 3-5 languages in the following areas:
It will always be a practical choice for those reasons. There are probably a lot more as well that I can’t think of at the moment.
This has to be a joke.
Yeah, thought the same. Python isn’t intuitive to install new packages.
Python is not the best at anything, but is the second best at almost everything. That’s the reason I use it. Everything I need to do, I know I can do it in python, maybe isn’t going to be the better or faster it could be, but it will work.
Yeah, Python is fast enough. Being assembly-fast is just not that important for a lot of things.
This. As a chemist I just need to do rather simple calculations, plots and sometimes use a package or two. My total yearly compute time is like 1 hour in python. Spending 5 years learning c, assembly, data science and what not just to reduce that to 1 minute is not really worth it.
I would add its a easy Scripting language. No compilation problem, richer that shell/bash makes it a powerfull choice.
And a really dont like it.
It is far more then that. It is a full up programming language.
I never understand why people think compilation is a barrier. But sure most python is not compiled.
If compilation takes more than a few single-digit seconds, IME, it breaks the development flow when you’re working on small fixes.
Having worked on large C++ projects, the solution to this issue is using a good IDE with an LSP. But it is quite a regrettable situation because setting up an LSP can be a difficult, and they also tend to break quite easily. Understanding how to make an LSP work is an important productivity skill in those kinds of positions.
And ccache or ninja. Something that reduces the amount of stuff that has be rebuilt.
Interesting, I had never heard of ccache before, though yes, all good build systems (CMake, Ninja, etc.) should cache intermediate object files.
But the projects I was working on were so large that even binary and unit test executables were so large that even they would take ~20 seconds to link. You can’t use caching to alleviate that buildtime cost unfortunately.
Python is largely compiled. All the pieces of a compiler are built into how it processes things. Almost nothing works on an interpreter model anymore, where each line is parsed and executed before handling the next. Unix-style shell scripts are one of the very few exceptions. I believe JavaScript also starts being interpreted in the browser in order to start executing immediately, but then a compiled version is swapped into the runtime as soon as it’s ready.
Depends on what you mean by compiled. Python typically translates to meta code and that is interpreted. True compilation to binary runs about 100x faster. Python is slow though there are faster versions and ways around it.
I mean that there are successive steps to transform the entire code into tokens, the tokens into an AST, and the AST into some intermediary or final form.
No, it doesn’t. Take a look at any of the number of projects that have attempted to compile Java to native code over the years. You’d be lucky to see any substantive gain at all. They sometimes have a use for packaging everything up in a single distributed binary, but you don’t do it for speed.
Things like C and Rust are fast because the language semantics can be compiled in a fast way.
We will have to disagree on that. This is all problem spectific, but I have found C code integrated via ctypes, cffi, or by a C extension is over 100x Python alone. Interestingly Python, Numba, and Numpy together which is a more pythonic solution can get to those speeds too.
All of the other approaches I have tried are much slower: Nuitka, Cython, Numpy alone, PyPy, etc.
To get best speeds one has to compile for your specific architecture and enable things like vectorization, auto parallel, and fast math. Most default builds including libraries do not do that.
Of course you did. Those are changing the semantics of the language. For example, things like Numpy store arrays more like how C does it than Python. That makes all the difference, not merely compiling to native code.
You can get about 10x by compiling Python using PyPy. So compiling is not nothing. Using Numpy alone is about 5x which surprised me. There is a lot of missleading stuff out there about how to make Python fast. Lot of people say CPython is pretty fast or that using a binary library like numpy is fast. No CPython is very slow and libraries are not always that fast.
Edit: Another compiler is Numba which is more specialized. It can get 30x on some code without numpy. Again compiling can help.
Python has essentially usurped BASIC.
Yeah, you can use it both for full applications (web or desktop) as well as simple scripts. The flow of getting from something simple to a full blown application is pretty smooth and natural - better than just starting out in Java or C++ and taking at least an hour before you get anything that runs.
The libraries part is enormous. Researchers just pull up math engines, database access is a breeze, and so on.
You got a specific need, there is a library for it. It’s like C/C++ without the hassle.
Python is popular because it’s whitespace based syntax make noobs think it’s easy to read. This and it’s pre-installation on Linux made it popular amongst academics who embraced it for data analysis. This lead to a lot of data scientists embracing it and writing libraries for it which created a virtuous cycle in that arena.
And it’s a damn shame because Python, and it’s whitespace nonsense and lack of type system, is a horribly impractical language for building anything at scale.
White space nonsense. Yes and no. I do not enjoy tracking down white space issues until I remember the alternative. Matching brackets, writing begin/end statements, and worst of all looking at code that appears well indented until later you find it is not. and it runs totally differently then it looks. Similar issues with semicolons.
Pretty much everyone uses LSEs these days so frankly matters less either way most of the time.
It does make it easier to read. We primarily use indentation to match brackets, so clearly, indentation is the most important readability feature between the two, not brackets.
Try reading the same piece of code: without brackets, and then without whitespaces and tell me which is better.
Whitespaces as syntax force you to indent your code properly and not be misleading.
Brackets are infinitely easier because any IDE (and most text editors) have a dead simple linting plugin which auto indents for you and lets you know when you fucked up. Your editor can barely do anything with whitespace only because everything is valid syntax.
Also lemme take this opportunity to say fuck yaml for exactly this reason.
I literally do that professionally, and I’m telling you that brackets are inarguably better.
You are used to whitespace; I have literally spent months refactoring and upgrading a react frontend and a python backend and am telling you that brackets are a single character and far more predictable, and then your IDE / Linter / Formatter can reliably use them to fill in the perfect indentation and whitespace.
My first professional language was Python, but I have since tried a half dozen others and would never choose to go back to a whitespace based language unless there was a good reason.
I’ve written C, JS, PHP, and a few others. Today I write Python almost everyday and wouldn’t trade it for any of them. Maybe Rust, but not because of brackets.
Instagram’s back bone is written in python. I’d say that’s pretty scaled.
No, it’s not. It was rewritten in Facebook’s version of PHP when they bought them.
Here’s a blog post from meta, they introduced immortal objects to python to help with their Instagram scaling.
https://engineering.fb.com/2023/08/15/developer-tools/immortal-objects-for-python-instagram-meta/
Huh, TIL, I didn’t realize they kept the gateway servers in Python.
Typing thing is way over blown. What did suck before Python 3 was indeterminate type result from the / operator. You never know if it would be integer or floating division looking at the code. Now it is floating.
It’s not made for scale. Use C/C++ or something if you want scale.
Or you know, TypeScript, Go, C#, Java, Swift, etc. etc.
Python isn’t great at anything except for having a lot of data science libraries already written for it.
Not only data science libraries, there is a library for about anything. Which is not the case for your list of languages.
Edit: not all languages in the list.
When you understand how python works, it’s quite simple to use too, and has lot less clutter than say java or c#.
Java has an excellent ecosystem for libraries. And a much more mature one in most cases than Python. The Spring framework alone is simply amazing.
Go’s ecosystem is pretty good as well and getting better.
The problem is that many python devs consider things like “type hints” and “classes” to be “clutter”. It takes a lot of discipline to write managable python. Being “newb friendly” is a curse. “stringly typed” crap does NOT scale well.
It can scale though. It parallelizes really well if you use queuing systems to distribute the load. You just have to make sure that the hot loops are in C/C++, and it is very easy to interface with compiled binaries via the C API.
So can assembly. But it takes a tremendous amount of discipline.
Python’s curse is that it’s popular with “newbs” who think it’s okay to use dictionaries for everything and that type hints are “clutter”.
Agreed. I have seen a lot of Python code that was really painful to massage back into a more structured object hierarchy. Java certainly does a bit better in that respect, and as a language, it does a much better job of encouraging better practices, but I think it’s also largely due to the kinds of people that use those languages as well.
Instagram runs on django.
It has optional typing
It has optional type annotations for use by linters*
Good point!