Hi, this might be an XY problem, so I will try to describe my issue shortly and then provide more optional background.
I’m a hobbyist and I often find myself building small projects where I combine personal and public data into something else, then I need to manually review it, sometimes make modifications, and then share it (in HTML, PDF, a calendar or something else). I do this by gluing together various tools (i.e: scrapy, sqlmodel, jinja, etc.).
I have used Django when it was still at a 1.X version and I’m now considering reusing it despite the fact that I don’t want to build a dynamic website. It will offer me a clear idea of how to split things, an ORM, a graphical interface (through its development server’s admin) and a way to output things.
It feels a bit wrong, so do you have any idea if there would be another kind of framework that I could use for such tasks?
More optional background
An example project would be:
getting a list of personal recipes with ingredients (i.e.: from a raw text file)
getting public data on the nutritional values for each ingredient (from a public database)
mixing everything with a weekly menu to get an idea if there is something wrong (ie: too much salt)
editing/filtering something, directly with the CLI, sometimes using rich or pandas to get a better view
being able to share this with my friends, take their recipes and output some other menus in HTML, PDF, etc.
all of this involves using click to build some CLI tools to automate the process
I’m a hobbyist so, even if those projects are a bit messy, the value I get from these is dependent on the amount of time I need to spend on this. I like to learn things about Python or other tools but my goal is mostly to use them.
Right now, to do such a project, I will need to use requests or scrapy to get the public data, a list of YAML/TOML/Json files or a ORM to store things, jinja or something else for the output. I’m spending a lot of time gluing everything from scratch each time.
You load your data into either plain python objects, or dataframes, or databases. You can create a notebook that loads data from files or from APIs when you initialise it. You can play with the data. There’s good visualisation options. And you can easily export to html.
However, I don’t think you can do very much via the CLI.
After following some of the tutorials of marimo, this is perfect. The worst part is that I was aware of the concept of notebook but never took the time to check it… It also resolves some issues about loading files, etc. The CLI look sufficient for my use case.
May I ask some related questions here, or should I open a new topic for each of these points? I tried to find the answer on the web but I can’t find something reliable…
what kind of data visualization library should I pick? plotly? altair? seaborn? something else? I want to create very simple plots, maybe some maps. The easier to learn and use will be the better. A nice output would be a bonus.
is there a python tool that would allow me to keep track of manually edited files and replay the edits? Like in the following?
take a source directory with some files in it (let’s say a friend of mine, sent me this)
manually rename some files, edit some lines in other files (my friend forgot to add the date somewhere or to properly format a line)
export the modifications as something replayable against the original directory (my friend sent me a new version of the original directory and I don’t want to make all the manual modifications one more time)
Python’s difflib is too low level for that. Maybe I just should use git instead?
Your friend and you should really use Git if you plan to contribute to the same project.
Using Github, Gitlab or others will act as a back up and you also have a history.
marimo has a UI for creating graphs, and for turning those graphs you’ve created into python code. It appears whenever you have a cell that has dataframe output (certainly for polars dataframes) and it creates altair graphs. So I’d start there.
I quite like plotly too TBH.
And I’d just use plain git too. You need to memorise about 6 commands to make good use of git. But I think it’s worth it.
On the other hand if you want to share a project with a friend, you kinda need to set up an account on github or gitlab and couple that account to git on your PC, and I recall that process is an extreme pain.
