Python, Org Mode, and writing Org tables to CSVs so that I can read them back
| emacs, org, writing, babel, pythonI’ve been getting deeper into Python so that I can model our personal finances. I really like using the pandas library to manipulate data. All those years I spent trying to juggle increasing complex spreadsheets… Working with Python code in Org Babel blocks is just so much more fun. I like being able to keep my assumptions in tables without having to fuss around with naming cells for easy-to-read formulas, slice and summarize parts of my data frames, organize my notes in outlines and add commentary, and define more complicated functions that I don’t have to squeeze into a single line.
I haven’t quite been able to tempt W- into the world of Org Babel Python blocks. Still, I don’t want to give up the awesomeness of having pretty tables that I can easily edit and use. So I have a bunch of named tables (using #+NAME:), and some code that exports my tables to CSVs:
#+NAME: tables| Table | Key ||---------------+---------------------|| assets_w | Description || assets_s | Description || tax_rates | || disposition | Asset || probate_rates | Asset || basic | Client information || base_expenses | Category || general | General assumptions |#+begin_src emacs-lisp :results silent :var tables=tables :tangle no(defun my-tbl-export (row)"Search for table named `NAME` and export."(interactive "s")(save-excursion(goto-char (point-min))(let ((case-fold-search t))(when (search-forward-regexp (concat "#\\+NAME: +" (car row)) nil t)(next-line)(org-table-export (format "%s.csv" (car row)) "orgtbl-to-csv")))))(mapc 'my-tbl-export tables)#+end_src
and some code that imports them back in, and formats tables nicely if I’m displaying them in Org. The in_org block doesn’t get tangled into index.py, so I don’t clutter command-line use with Org table markup.
#+begin_src python :results silent :tangle noin_org=1#+end_src#+begin_src python :results silent :exports codeimport pandas as pdimport numpy as npimport orgbabelhelper as obdef out(df, **kwargs):if 'in_org' in globals():print(ob.dataframe_to_orgtable(df, **kwargs))else:print(df)return df#+end_src#+begin_src python :results silent :var tables=tables :colnames yesfor row in tables:table = row[0]index = row[1]if row[1] == '':index = Noneglobals()[table] = pd.read_csv(table + '.csv', index_col=index).apply(pd.to_numeric, errors='ignore')# print(globals()[table])#+end_src
Then I can use C-c C-v C-b (org-babel-execute-buffer) to update everything if I change the table in my Org file, and I can use C-c C-v C-t (org-babel-tangle) to create an index.py that W- can read through or run without needing Org.
2 comments
Peter
2019-11-01T18:49:25ZI love to hear more about how you are modelling your finances, what other tools are you using? What are you hoping to model?
sachac
2019-11-05T19:21:00ZDoing estate planning at the moment. We want to make sure we can cover final expenses, ideally without having to sell the house at a difficult time, that the survivor has enough to support themselves and A- until A- is old enough, and that J- also gets a fair share. Blended families mean complex planning. :) I also track my finances with the command-line Ledger program, which is awesome.