Julia Community 🟣: ndx

JuliaLang? Your complains have neat solutions.

ndx — Thu, 25 Aug 2022 07:33:00 +0000

Let me give some of the major reasons people complain about Julia and possible ways to solve them.

Perceived Problem 1: There are little to no questions on Julia' s stackoverflow tag, so if you have an issue, you'll have a hard time to solve it.
- Solution: The new users don't know Julia is currently a community project (no big tech company is currently behind it unlike other languages), and that the community is very active on https://discourse.julialang.org, official julialang slack and is actively welcoming discussions, issues and pull request on GitHub. If they knew this, they would know their questions would get responses (and answers) in at most 30mins on any of those official Julia channels.
Perceived Problem 2: Python has more libraries than Julia. In Julia, there are little libraries in the ecosystem, and they're not actively managed.
- Solution: Put your expectations in check, programming languages and softwares don't work that way. Real world applications take time and they move in trends. Using python as an example, below are the most popular libraries you would mostly use if you use python a lot:
  - pypy a just-in-time compiler for python to help speed up the language published an initial release in 2007, 16 after python was created.
  - cython an extension of python for those who want to speed [python] up published an initial release in 2007, 16 after python was created.
  - Pandas the most popular python library for dataframes published an initial release in 2008, 17 years after python was created.
  - Numpy the most popular python library for numerical computing published an initial release in 2006, 15 years after python was created.
  - Matplotlib the most popular python library for plotting published an initial release in 2003, 12 years after python was created.
  - Tensorflow one of the most popular python library for ML and AI published an initial release in 2015, 24 years after python was created.
  - Pytorch one of the most popular python library for ML and AI published an initial release in 2016, 25 years after python was created.
  - Scikit-learn one of the most popular python library for ML published an initial release in 2007, 16 years after python was created.
  - Selenium one of the most popular Python library for browser automation published an initial release in 2002, 11 years after python was created.
  - Django one of the most popular python web framework published an initial release in 2005, 14 years after python was created.
  - Flask of the most populat python web framework published an initial release in 2010, 19 years after python was created. None of the libraries were developed 10 years after python. Any genuine person would ask this: Then what were people doing with Python for the first 10+ years… I guess they were using Perl, BASIC and bashing on Python as a lot of people are currently doing to Julia. So if you claim to love python and hate julia because of libraries availability, you're just a user who loves python because its easy to use with add-ons (libraries), not necessarily the language design itself; and moreover you would have said same thing about python had you come into it exactly 10 years after the language was developed (just like you say about Julia now).
  Now contrast the above with how Julia is evolving:
  - DataFrames.jl one of the most popluar julia library for tabular data published an initial release in 2013, 1 year after Julia was created.
  - Plots.jl one of the most popular julia library for plotting published an initial release in 2015, 3 years after julia was created.
  - UnicodePlots.jl one of the most popular julia Unicode-based scientific plotting for working in the terminal published an initial release in 2015, 3 years after julia was created.
  - Makie.jl one of the most popular julia high level plotting on the GPU published an initial release in 2018, 6 years after Julia was created.
  - Flux.jl one of the most popular julia ML library published an initial release in 2017, 5 years after julia was created.
  - Knet.jl one of the most popular julia deep learning framework published an initial release in 2016, 4 years after julia was created.
  - Franklin.jl one of the most popular julia static website generator published an initial release in 2019, 7 years after julia was created.
  - Genie.jl one of the most popular julia web framework published an initial release in 2018, 6 years after julia was created. There are over 8000 registered packages in the Julia ecosystem, so if you care to search well enough, you might find what you need. 10 years into a language and they're already over 8000 registered packages. This should tell you how fast the language is evolving.
The above comparisons are not given to say Julia is better than Python. The message it's trying to convey is that softwares and especialy open source softwares takes time. Since no one is getting paid for what they do, why do you think it will get out of the door just with a winkle? It takes time, so have that in mind. Anyways, if you want a library to be developed instantly for you, then talk to Julia Computing (but be ready to pay for it as its not free).

All the growth of those python libraries came as a trend of the growth in ML and AI, and python was the only simple and versatile language dominant among users to glue them to at that time - so you see the trend now. The libraries weren’t developed because of Python (most of the development were done in C++ and C, not in python as we all know), they were only glued to Python so users can easily call them. There are more libraries you know of in Python than the ones listed above, chances are they were initially released 20+ years after Python already existed. So yes Julia don’t have all the libraries you might need now, but its just 10 years old and chances are it has enough of the libraries to do most of the stuff you would need now (except in very niche cases where you either have to develop one yourself or wait for others interested in what you do to develop it as time goes on).
Perceived Problem 3: There are a lot of tutorials out there to learn other languages, Julia's tutorials on the internet is very small.
- Solution: Julia's documentation isn't the best you'll find on earth, but it does a good job of explicitly telling you what and what you need to know and master to effectively use the language. If you have a problem the docs doesn't address, then instantly ask questions on any of the channels highlighted in the solution to problem 1 (you're guaranteed of an answer in 30mins) and also pull a request for it on github and you're sure to get a review in a day (at most). Contrasting it with a language like python again, the tutorials boom on python only started gaining momentum 20 years after python already existed, as a result of the growth of ML and AI; and it didn't happened overnight to get to where it is, it took time.
Perceived Problem 4: Julia is 1-based indexing and the other languages are all 0-based indexing, so its hard to get used to this new behaviour and it will slows my productivity.
- Solution: Julia is a isn't the only language using 1-based indexing. FORTRAN, APL, MATLAB, R, Wolfram, Lua, COBOL, etc., all use 1-based indexing and they have millions of developers using these languages and great softwares out there (so you see its not much of a big deal as most people would try to make it). Within a month of development in Julia, you will probably get used to it. If you're still keen on using 0-based indexing, then JuliaArrays provides OffsetArrays.jl for you, so you can use 0-based indexing and even negative indexing as you wish to in Julia.
Perceived Problem 5: In other langauges I can quickly build standalone binaries, this is hard or not even possible in Julia.
- Solution: Honestly, there is work going underground for this and its best left to say "NO Julia can't build standalone binaries for now". Of course, there are multiple workarounds like PackageCompiler.jl if you currently wish to build one still, but it's not yet effective like other languages have.
Perceived Problem 6: In python for example I can easily call C and C++ libraries, this is hard or not even possible in Julia.
- Solution: You can do same and possibly even more in Julia. With Julia you can call most programming languages and use their code and libraries effectively.
  - For Python, you can use PyCall
  - For C, you can use ccall
  - For C++, you can use Cxx and CxxWrap
  - For R, you can use RCall
  - For Java, you can use JavaCall
  - For Mathematica/Wolfram, you can use MathLink
  A lot more others showing you how to use other languages from Julia can be found here: JuliaInterop

To conclude, I will add this: If a language/environment is “good enough” for someone, no amount of persuasion will convince them to switch. In my experience, people who gravitate to Julia do so because it really solves the problems they had with other languages (in terms of speed, elegance, abstraction, "two-language problem"), or expect that it will, given time, so it is worth investing in.

Do you have a datatype, but keep googling the functions to work with it? Try This!!!

ndx — Thu, 28 Jul 2022 00:37:37 +0000

Someone posted this in the Julia reddit page Pains of Julia compared to Python. In that article, he listed out 7 things he loved in Python but couldn't find in Julia. I found interest in number 7, because my first few weeks in Julia gave me same pain (which would later be the reason I loved the language more). It states, reading from the article:

7. Lack of OOP. Julia is strictly a functional programming language - there are no objects. So, if you want to perform operation on a datatype, you have to remember! (or google) the name of the function. For example, if you have a graph datatype named my_graph, in a language with OOP (such as python), you can type my_graph. and your IDE can suggest functions available for the my_graph object, such as my_graph.topo_sort(). In Julia, you will have to remember (or google) topo_sort function is available to process graph datatype and will have to type topo_sort(my_graph). This is the part I am having the hardest time getting adjusted to and has been a major cause of slow down.

I won't explain if Julia is OOP or not, but focus on the other part that has to do with remembering functions to work with in the datatype. Now, I'm sure a lot of user had same pain and I'd love to know how you conquered it. Meanwhile, let me share my own experience of how I solved the puzzle in a way that got me used to the langauge, where I don't need much googling, but rather explanations on specific implementations of some methods I might not understand, which the Julia community on slack and discourse are happy to help with.

Honestly, if you haven't been told yet, then hear it clearly now: Julia would be a bone in your throat if you hate using the REPL it provided. A bone in your throat, because you would know what to do a lot of the time, but don't know how to do it in Julia.

You can solve this using the steps I stated in the reddit answer I gave to the thread.

Well now lets see its implementation here on Julia forum.

The mandatory step is to download Anaconda, VS Code and set it up for Jupyter Notebook and Julia. Just google the workarounds for setting this steps up and ask questions politely (in any one of Julia forums) if you're stuck, then in 1 hour you should be done.

Hopefully you're done with the steps above and now it's time to work into the magic that helped me get familiar with Julia and understand most of its methods.

Open Up Your VS Code

From the image's description, press ^ and \ to toggle the terminal or use ⌘ + J keys to toggle panel. The screen should now look this way.

Right click on Terminal and then from the dialog-box opened, click on Move Panel Right.

If you have done so, then you screen should now look this way.

Now copy the execution path of Julia on your local computer and paste it into the terminal on VS Code. An easy way to get the execution path is to run the Julia REPL and then copy the execution path from the command line. The execution path as seen in the picture below is the text circled on yellow borders.

If you've done so then your VS Code terminal should look like the image below.

We all know how to set up the terminal for VS Code now and have it run our Julia REPL, making us have it all in one environment.

Open a new Julia file from the code editor in VS Code (use ⌘ + N keys) and click on open a different editor as shown in the image below.

Select the Jupyter Notebook from the drop-down menu as shown in the image below.

Your VS Code editor should now look like this.

Now go into the cells of the notebook and type in this function as shown below.

function methods_inside(modules...)
    for m in modules
        println($(uppercase(m)))
        println.(names(m))
        println()
    end
end

Now, at this point you should know the Core, Base and InteractiveUtils modules are considered the standard libraries of the Julia programming language; meaning you can't do any meaningful work in Julia without them. So all methods exported by the standard libraries are the methods provided by Julia.

The names method used in the function defined above is used to get an array of the names exported by a Module, excluding deprecated names.

So we will pass those 3 modules of the standard libraries as arguments to our function we created. Type the code below into the next cell of your notebook.

methods_inside(Core, Base, InteractiveUtils)

If you have done the above, your editor should now look like this.

Click on in a text editor to get the whole text, because text files are truncated to fit into the screen by the default settings in VS Code (you can change this if you want). Your VS Code editor should now like this.

You now have all the methods in the standard libraries in a text editor. You can scroll down and then see the methods listed in Base module and InteractiveUtils as well. Your laser-like focus should be on the Base first and then work your way back to the Core and then the InteractiveUtils section of the methods (its not a rule, that's how I did mine and it worked for me, so change it to fit your needs).

The question next is: what do we do with this? Well the answer is what I did with it (as that is where my knowledge is limited to), as I can't tell you how you would use it.

Save the text file that has all the methods with a .txt file extension (so it can be opened up in VS Code), then ALWAYS keep it open in your VS Code environment when working with Julia as well as the Julia REPL on the terminal in the right hand side, this would make your workflow so swift.

I saved mine with All Methods.txt. So each time I open up VS Code for working with Julia, I could easily lookup methods and then use the help?> entered by typing ? on the Julia REPL to quickly find out their meanings all in one environment.

I know this might sound stupid, but I usually do a very quick glance through the All Methods.txt file before writing my own methods for things, a lot of the time Julia has defined what I want executed. An example is checking if all the elements in an interable is distinct from each other. One would think a simple for...end loop is an easy way to solve this. Of course, it is, but glancing through the file, I see the allunique method, which sounds like what I want. Then I look up its documentation on the help session in the REPL and immediately I know its what I want. Would google had helped me? Probably yes/no, but whatever the answer may be, using this approach keeps making me familiar to the standard library of the language.

Another method is checking the perfect square less than or equal to a number. One would easily jump into coding, but trust me, as simple as this function sounds, Julia has provided it for you.

Using the built-in functions, means your code would be more Julian, making its intents readable among the Julia community and run faster as well. So please use it.

On my study and free-time as well, I also explore new methods. Julia is so new, so a lot of questions which can help reveal new methods or ways of doing things have not been asked yet on stackoverflow or whatever, so googling won't help now. Your best bet is to learn them yourself and write more syntactic Julian code thereby making your code more scientific and program run more faster.

How did you solve yours, I'd love to know?

Where do we go next from here with programming languages?

ndx — Tue, 12 Jul 2022 16:43:02 +0000

Once upon a time - exactly 74 years ago - a man born in Berlin, Prussia, at the time of the German Empire, published the first concept of what we call today as a "high-level programming language". But wait, a lot happened before then.

History has it that after he graduated, he went to work for one of the big motor companies of his time, using his artistic skills in the design of advertisements. He also worked as a design engineer at one aircraft factory in a place near Berlin. This required the performance of many routine calculations by hand, leading him to theorise and plan a way of doing them by machine.

This man, who later in the twilight of his days on earth would die from a heart attack, was a self-taught, and built and developed a lot of his works without knowledge about other mechanical computing machines that existed already. Although he would later go on to build on existing discoveries and technologies, between 1936 and 1945 he was in near-total isolation; he completed his work entirely independently of other leading computer scientists and mathematicians of his day.

It was in 1945, after the Luisenstadt bombing, that he fled from Berlin to the rural Allgäu. In the extreme deprivation of post-war Germany, he was unable to build computers. Unable to continue building computers - which was also forbidden by the Allied Powers - he devoted his time to the development of a higher-level programming model and language.

In 1948 he published a paper and presented it at one of the annual meetings of his time. His work failed to attract much attention. In a 1957 lecture, he expressed his hope that the higher level programming language he had created, "after some time as a Sleeping Beauty, will yet come to life." He expressed disappointment that the designers of another programming language, who used most of his works never acknowledged the influence of high level programming language on their own work.

The man we've been taking about is Konrad Zuse, and Plankalkül was the name of the higher level programming language he built, which was essentially made for engineers. ALGOL 58 was the programming language designed with most of his work, but the developers never acknowledged the influence of Plankalkül on it.

Plankalkül was more comprehensively published[vague] in 1972. The first compiler was implemented by Joachim Hohmann in his 1975 dissertation. Other independent implementations followed in 1998 and 2000 at the Free University of Berlin, history continues of course, but we will end there for now.

To know where we are going we somehow need to know where we are coming from - and that story was introduced just as a way to take us back a little to see where this "higher level programming language" thing started and where it's likely to go.

There are currently more than 700 programming languages out there as at this time of writing. Some are low level, most are high level. Some designed for speed, others designed for simplicity. Some designed for security, others designed for low-level interactions, and the list goes on and on and on.

Go out now and ask someone with no computer programming language: how many programming languages do we have? Let them make an educated guess with the knowledge of how they see the computer world, chances are they would all answer in the range of 0 - 10. The ones who answer in the range of 20 - 50 would be waiting to get a response back from you that he/she was wrong. Chances are as well, that if you should make an educated guess now, you might get it completely wrong as well. The vague answer is that there are over 700 programming languages. WOW!!!!

I failed it as well, it was only by "googling" that I came to know of this unimaginable number. Now, the question is: why 700? why not 1? why not 5? why not 10? why not even 20? how come its over 700?😱. That's an open question, so discuss it with your friends.

My view is this: there are over 700 but are all 700 being massively used. Of course, that's a simple NO. If all 700 are not being used, why do they exist? They exist because there's an informal law in computer programming: "programming languages are only created, they cannot be destroyed."

But why can't they be destroyed? Because the trend is: learn programming and then fall in love with a language. Then technology advances and you know we can no longer use your "loving language" efficiently anymore as the current world demands something with a different philosophy, which your loving language can do if tweaked, but wasn't designed for, but you choose to continue using it and only use a different language when your job requires it. So you see, we can only create but not destroy, because every creation gathers a fan base which won't let it die, no matter its age - Plankalkül is a case in point, as its still being used.

But that doesn't answer the question of this article - where do we go next from here with programming languages? Honestly, speaking technology is one sweet field where advances are great (and they would always come), only if it won't hurt or demand you to learn something kinda entirely new or change your current perspectives and beliefs. The golden question now is this: if the man/woman who created your programming language "loved" the language they learnt programming with so much that it blinded them to see a change or think in a new paradigm, do you think we would have seen another high-level programming language other than Plankalkül ?

Would there have been Python, if Van Rossum so loved BASIC, or would there have been C++ if Bjarne so loved C, or would there have been Julia if Viral could only see Lisp, and the list goes on and on and on. The point is technology itself is built on ADVANCEMENT, so no matter how much you love a language, know its only going to be for a while or if you're lucky enough, your whole lifetime; but no matter how hard you try, your kids won't use what you use, or even if they used what you use, they won't use it in the same way that you used it - that's how technology works, so embrace it.

But still the question is not answered yet - where do we go next from here with programming languages? Where we go next from here depends on two things:

A body would be set up to regulate the creation of programming languages.
New languages won't be created except they're coming with a completely different philosophy.

Now think for a moment if advancement and discoveries was regulated? Would there be any "outside-the-box" discoveries accepted. Of course the answer is NO. History shows how regulation only helps in maintaining moral conducts and order but kills imagination and discoveries into new areas; this is the reason our school system still thinks "grading students" has been the norm and so is good, even when mathematically and statistically speaking, grades are no projection into the future of any child, as countless evidences has shown, and rather than helping students, it kills their morale and makes some feel like losers. The truth is if there was a regulation (which there won't be) in the technology space to control new discoveries, the world would have been a boring place, as we would have only seen ways on how to improve the post office system instead of envisioning an "email" along with it.

C++ was designed with "classes" in mind. So its built on the already existing ideas of "C" but with "classes" included. So can we say "classes" is a whole different philosophy to enforce the creation of a new language. If its not, why don't we just extend the language "C", so it accepts the "class" idea and then we don't need a new language, but simply have one with two design philosophy? The answer to this question is a question: a car can drive in a blazing speed and take us where we want to in a small time and with comfort compared to if had to walk by foot. The aeroplane can fly us to where we want to in a small time and with comfort compared to if we had to walk or drive a car. But if a car "can take us in a small time and with comfort" to where we want compared to if we walk by foot, why do we need an aeroplane then? Why not just add to the design of a car so it can fly, and then we only need one device, just a car? Until you can answer this question with the car and aeroplane analogy, you can't answer the question with C and C++.

The truth is "design and philosophy" is very different from "features and abilities". "Features and abilities" build on pre-existing "design and philosophy", but then "design and philosophy" are completely built on new "ideas and thought process." There is no difference between a Tesla car and a Ferrari, except the features and abilities. So we only needed a Ferrari when we needed new features, for this we do not need a new device, we simply have to build on the "pre-existing design and philosophy" of a car. But when it comes to an aeroplane, it brings a new idea - the idea of objects being able to fly in the air. That's a whole new "though process". So you see, its best if we come down and think of how to make "objects" fly instead of how to make a "car" fly. Then we finally did it, the name doesn't matter - we could have called an aeroplane a "flying car", what matters is the different idea an aeroplane has. This is how it is too with programming languages.

Yes, there are languages with "features and abilities" which in the true sense the developers should have thought of contributing to the orginal languages they borrowed the idea from, instead of creating a new one - but we humans know how we all want something called "our" own, instead of I built "their" own: this is the reason there are many programming languages in the world today, not the design and philosophy thing. Its good to know C++ was designed for classes and C for procedural. So when a use case comes for each design, I know where to go to.

The question then is if we have very limited programming paradigms and concepts, how come we have over 700 languages as if we're on planet talking to humans, when in its true sense we're only talking to a computer? The reason is the "features and abilities" that leads to new programming languages instead of developing already existing ones when its just "features and designs". Of course we all caused it, when we started attributing programming languages to "owners", "creators", etc. If there was no "creator" or "owner" to a programming language, then the idea of contribution would have been easy for us humans. Look at the earth, imagine if only Mr. A owned planet earth, we would all have seen this has a place to just dump and go, but because we see it as for "humanity" as a whole, we all contribute to its good. So this is one area in the future where open source can go to, an area where there are only "contributors" and no "creators", "owners" or "founders" exists.

Still the question yet is not answered: where do we go next from here with programming languages. I think now I can answer in my view. The programming languages numbers would keep surging at a rate probably 100 against new programming paradigms or concepts, as the creation of any programming paradigm leads to at least 100 programming languages with same "design and philosophy" but different "features and abilities", i.e, programming languages : programming paradigms is at a ratio of 100:1.
Secondly programming paradigms and new concepts would be getting diminished as a lot of emphasis would be on languages instead of new paradigms that would help the technical computing community. This is of course is my view and it can be wrong.

Finally I want to speak on Julia vs Python? Comparing this two languages is like comparing an aeroplane to a car. They both have one purpose: take us to where we want to in quick time and comfort than if we would work. But with different "design and philosophy", one moves on the land and is quick compared to a human foot, the other moves on the air, and is quickest compared to the car and a human foot.

So you sit there and ask? Should I switch to Julia from Python, or go learn C++, or even ruby. C++ is a programming language used to talk on a lower-level with the computer. It is still popular because of an idea I call "diminishing-increasing dependency". Diminishing-Increasing dependency is a concept where we are going away from something, but haven't found other alternatives to fully replace it, so the more we go away from it looking for other alternatives, the more we rely on it. Check the last 100 programming languages designed, and you'd see what I'm talking about: the syntax is so different from C++, that you wouldn't even think any programming language should look like C++ do.

But the truth is, yes the generation we're slowly moving into does not want a program to be "dirty" like C++ even when it does the work, we want something we can read to somebody else and still give the computer to execute it. But the truth is, the computer is still at a stage where it only understands 0's and 1's - so C++ is the language that interacts with it so closely at that level, hence the more better a language we look for, the more C++ we would be using. So C++ would keep getting popular as it has no competitor, because no languages are being develop that way to be "dirty" anymore, just so it can talk on a lower-level with a computer. So you see modern languages having more competitors than old languages like FORTRAN, COBOL, C and C++, because no one is building languages in that style anymore - and the more we look for the perfect language, the more we have to depend on the only ones we have for now that can talk to a computer in a very close way - "Diminishing-Increasing Dependency".

Of course, 100 years from now, C++ and its counterparts would all be gone in a way and would only be used as legacy languages, because we would have developed the computer to a stage where it can take human-readable code and still be fast and even faster than C++ and its counterpart would have been - Is 100 years too much an exaggeration? Well, just give it any number, but one thing is sure, its not happening in your time if you're old enough to read this as at the time of writing.

Now back to Python and Julia. If you're an avid Python lover, do you belief deep down 20 years from now, we would all still be depending on Python to run numerical simulations? Well the future would tell anyway, but sure we know the answer in a way.

The truth is we can't depend on Python for numerical simulations 10, 20, 30 years from now. The language wasn't designed for "speed" and "numerical computing". It was designed for "readability" and "general use". Every day, data is being flooded out into the space, and we want something a bit faster and one that makes lives easier as well. Python is a great language but it just got into a community that would come back to bite it. Numerical computing is not "general use", it needs to be "fast", and that's a first requirement. Then it needs to embrace the mathematical world and breathe it. Python itself does not have any one of these traits, it only works as an interface after the work is done inside by other core languages - making you worry over learning another language if you have an idea that's worth building from scratch and is very time intensive.

Now when you come to Julia, you could in a way exaggerate that the 100 years I said above is finally here. It gives you the speed and even more than the speed C++ would give you (this is still largely debatable) along with the readability. So in a sense, Julia is the 21-st century language, as it seeks to finally make C++ and its counterpart obsolete. So you see, Python is no comparable to Julia, Julia is competing with C++ and counteparts we all know that, but as for Python, it is just too small for it - the fact that the community hasn't fully accepted Julia yet doesn't makes Python anywhere near it; Julia is just waiting for full industry acceptance. Python is only there because of its uncountable libraries available (libraries mostly developed outside the language f its does a lot of intensive work) and community base it has over Julia, not the language itself

Julia beats C++ speed on some instance and also breathes and lives the math world. You can use '√' to represent a square root and its valid code. 2x - 5y + 4x * 23y is valid Julia code. This is just a pinch of how it lives for the mathematical world - so your kids would grow up to see that just as we understand "2x - 5y", the computer understands it as well. Of course, python can add that if it chooses, but the point is, it wasn't designed for that - you want to be on an aeroplane flying and not a car that was tweaked so it can fly.

Yes, there are Python lovers and they won't change no matter what. There are C++ lovers and they won't change no matter what. But whether you choose to change or not, its clear - JULIA IS THE 100 YEARS WAY FROM NOW.