<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet href="/assets/atom.xsl" type="text/xsl"?><feed
	xmlns="http://www.w3.org/2005/Atom"
	xmlns:thr="http://purl.org/syndication/thread/1.0"
	xml:lang="en-US"
	><title>Sacha Chua - tag - babel</title>
	<subtitle>Emacs, sketches, and life</subtitle>
	<link rel="self" type="application/atom+xml" href="https://sachachua.com/blog/tag/babel/feed/atom/index.xml" />
  <link rel="alternate" type="text/html" href="https://sachachua.com/blog/tag/babel" />
  <id>https://sachachua.com/blog/tag/babel/feed/atom/index.xml</id>
  <generator uri="https://11ty.dev">11ty</generator>
	<updated>2015-01-30T21:00:00Z</updated>
<entry>
		<title type="html">Getting data from Org Mode tables</title>
		<link rel="alternate" type="text/html" href="https://sachachua.com/blog/2015/01/getting-data-org-mode-tables/"/>
		<author><name><![CDATA[Sacha Chua]]></name></author>
		<updated>2015-01-27T00:21:05Z</updated>
    <published>2015-01-30T21:00:00Z</published>
    <category term="emacs" />
<category term="org" />
		<id>https://sachachua.com/blog/?p=27871</id>
		<content type="html"><![CDATA[<p><a href="http://orgmode.org/">Org Mode</a> is an amazingly powerful package for Emacs. I&#8217;ve been learning a lot about how to use its support for plain-text tables and spreadsheet calculations.</p>
<div id="outline-container-unnumbered-1" class="outline-3">
<h3 id="unnumbered-1">Using table data in Emacs Lisp with the :var argument</h3>
<div id="text-unnumbered-1" class="outline-text-3">
<p>For example, I wanted to be able to define my abbreviations in an Org Mode table in my <a href="https://sachachua.com/dotemacs">config</a>. I remembered coming across this technique a few weeks ago, but I couldn&#8217;t find the webpage with the code. It turned out to be simple to write from scratch. Here&#8217;s the plain text I added to my config.</p>
<div class="org-src-container">
<pre class="src src-org"><span class="org-org-meta-line">#+NAME: abbrev</span>
<span class="org-org-table">| Base  | Expansion                             |</span>
<span class="org-org-table">|&#45;&#45;&#45;&#45;&#45;&#45;-+&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;-|</span>
<span class="org-org-table">| bc    | because                               |</span>
<span class="org-org-table">| wo    | without                               |</span>
<span class="org-org-table">| wi    | with                                  |</span>
<span class="org-org-table">| ex    | For example,                          |</span>
<span class="org-org-table">| qm    | sacha@sachachua.com                   |</span>
<span class="org-org-table">| qe    | </span><span class="org-org-link">http://sachachua.com/dotemacs</span><span class="org-org-table">         |</span>
<span class="org-org-table">| qw    | </span><span class="org-org-link">http://sachachua.com/</span><span class="org-org-table">                 |</span>
<span class="org-org-table">| qb    | </span><span class="org-org-link">http://sachachua.com/blog/</span><span class="org-org-table">            |</span>
<span class="org-org-table">| qc    | </span><span class="org-org-link">http://sachachua.com/blog/emacs-chat/</span><span class="org-org-table"> |</span>

<span class="org-org-block-begin-line">#+begin_src emacs-lisp :exports code :var data=abbrev</span>
(mapc (<span class="org-keyword">lambda</span> (x) (define-global-abbrev (car x) (cadr x))) data)
<span class="org-org-block-end-line">#+end_src</span>
</pre>
</div>
<p>The <code>:var data=abbrev</code> argument to the Emacs Lisp source block is where all the magic happens. Here, it takes the data from the table named &#8220;abbrev&#8221; (which I set using <code>#+NAME:</code> before the table) and makes it available to the code. Emacs evaluates that data when the code is tangled (or exported) to my configuration. The code that&#8217;s in my <code>Sacha.el</code> looks like this:</p>
<div class="org-src-container">
<pre class="src src-emacs-lisp"><span class="org-rainbow-delimiters-depth-1">(</span><span class="org-keyword">let</span> <span class="org-rainbow-delimiters-depth-2">(</span><span class="org-rainbow-delimiters-depth-3">(</span>data <span class="org-rainbow-delimiters-depth-4">(</span>quote <span class="org-rainbow-delimiters-depth-5">(</span><span class="org-rainbow-delimiters-depth-6">(</span><span class="org-string">"bc"</span> <span class="org-string">"because"</span><span class="org-rainbow-delimiters-depth-6">)</span>
                    <span class="org-rainbow-delimiters-depth-6">(</span><span class="org-string">"wo"</span> <span class="org-string">"without"</span><span class="org-rainbow-delimiters-depth-6">)</span>
                    <span class="org-rainbow-delimiters-depth-6">(</span><span class="org-string">"wi"</span> <span class="org-string">"with"</span><span class="org-rainbow-delimiters-depth-6">)</span>
                    <span class="org-rainbow-delimiters-depth-6">(</span><span class="org-string">"ex"</span> <span class="org-string">"For example,"</span><span class="org-rainbow-delimiters-depth-6">)</span>
                    <span class="org-rainbow-delimiters-depth-6">(</span><span class="org-string">"email"</span> <span class="org-string">"sacha@sachachua.com"</span><span class="org-rainbow-delimiters-depth-6">)</span>
                    <span class="org-rainbow-delimiters-depth-6">(</span><span class="org-string">"dote"</span> <span class="org-string">"http://sachachua.com/dotemacs"</span><span class="org-rainbow-delimiters-depth-6">)</span>
                    <span class="org-rainbow-delimiters-depth-6">(</span><span class="org-string">"web"</span> <span class="org-string">"http://sachachua.com/"</span><span class="org-rainbow-delimiters-depth-6">)</span>
                    <span class="org-rainbow-delimiters-depth-6">(</span><span class="org-string">"blog"</span> <span class="org-string">"http://sachachua.com/blog/"</span><span class="org-rainbow-delimiters-depth-6">)</span>
                    <span class="org-rainbow-delimiters-depth-6">(</span><span class="org-string">"ec"</span> <span class="org-string">"http://sachachua.com/blog/emacs-chat/"</span><span class="org-rainbow-delimiters-depth-6">)</span><span class="org-rainbow-delimiters-depth-5">)</span><span class="org-rainbow-delimiters-depth-4">)</span><span class="org-rainbow-delimiters-depth-3">)</span><span class="org-rainbow-delimiters-depth-2">)</span>
  <span class="org-rainbow-delimiters-depth-2">(</span>mapc <span class="org-rainbow-delimiters-depth-3">(</span><span class="org-keyword">lambda</span> <span class="org-rainbow-delimiters-depth-4">(</span>x<span class="org-rainbow-delimiters-depth-4">)</span> <span class="org-rainbow-delimiters-depth-4">(</span>define-global-abbrev <span class="org-rainbow-delimiters-depth-5">(</span>car x<span class="org-rainbow-delimiters-depth-5">)</span> <span class="org-rainbow-delimiters-depth-5">(</span>cadr x<span class="org-rainbow-delimiters-depth-5">)</span><span class="org-rainbow-delimiters-depth-4">)</span><span class="org-rainbow-delimiters-depth-3">)</span> data<span class="org-rainbow-delimiters-depth-2">)</span> <span class="org-rainbow-delimiters-depth-1">)</span>
</pre>
</div>
</div>
</div>
<div id="outline-container-unnumbered-2" class="outline-3">
<h3 id="unnumbered-2">Looking up data with org-lookup-first, org-lookup-last, and org-lookup-all</h3>
<div id="text-unnumbered-2" class="outline-text-3">
<p>You can do more complex things with Org tables, too. Inspired by Eric Boyd&#8217;s talk on his <a href="http://quest.digitalcrusader.ca/">Epic Quest of Awesome</a> (which he based on <a href="http://nerdfitness.com/blog/epic-quest/">Steve Kamb</a>&#8216;s), I started putting together <i>my own</i>. I made a list of little achievements, guessed at the years, and assigned arbitrary experience points.</p>
<p>The <code>achievements</code> table had rows like this:</p>
<table border="2" frame="hsides" rules="groups" cellspacing="0" cellpadding="6">
<colgroup>
<col class="right">
<col class="left">
<col class="right">
<col class="left">
<col class="left"> </colgroup>
<thead>
<tr>
<th class="right" scope="col">Approximate date</th>
<th class="left" scope="col">Category</th>
<th class="right" scope="col">XP</th>
<th class="left" scope="col">Description</th>
<th class="left" scope="col">ID</th>
</tr>
</thead>
<tbody>
<tr>
<td class="right">2014</td>
<td class="left">Life</td>
<td class="right">50</td>
<td class="left">Became a Canadian citizen &#8211; <a href="https://sachachua.com/blog/2014/09/becoming-filipino-canadian/">link</a></td>
<td class="left">L_CAN</td>
</tr>
<tr>
<td class="right">2014</td>
<td class="left">Programming</td>
<td class="right">20</td>
<td class="left">Used NodeJS and AngularJS for a client project &#8211; <a href="https://sachachua.com/blog/2014/09/yay-rocked/">link</a></td>
<td class="left">P_NOD</td>
</tr>
<tr>
<td class="right">2014</td>
<td class="left">Programming</td>
<td class="right">5</td>
<td class="left">Pulled information out of Evernote</td>
<td class="left"></td>
</tr>
</tbody>
</table>
<p>I wanted to summarize the points by year: points gained, total points, level (according to a lookup table based on D&amp;D experience points), and description. The lookup table was structured like this:</p>
<div class="org-src-container">
<pre class="src src-org"><span class="org-org-meta-line">#+TBLNAME: levels</span>
<span class="org-org-table">| Total XP | Level | Adjective             |</span>
<span class="org-org-table">|&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;+&#45;&#45;&#45;&#45;&#45;&#45;-+&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;-|</span>
<span class="org-org-table">|        0 |     1 | trained-initiate      |</span>
<span class="org-org-table">|     1000 |     2 | experienced           |</span>
<span class="org-org-table">|     2250 |     3 | savvy                 |</span>
<span class="org-org-table">|     3750 |     4 | veteran               |</span>
<span class="org-org-table">|     5500 |     5 | unusually experienced |</span>
</pre>
</div>
<p>Now for the summary table. I created rows for different years, and then I used Org Mode to fill in the rest. (Org Mode! Wow.)</p>
<div class="org-src-container">
<pre class="src src-org"><span class="org-org-table">| Year | Points gained | Cumulative points | Level | Adjective        |</span>
<span class="org-org-table">|&#45;&#45;&#45;&#45;&#45;&#45;+&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;-+&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;-+&#45;&#45;&#45;&#45;&#45;&#45;-+&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;&#45;|</span>
<span class="org-org-table">| 1997 |             0 |                 0 |     1 | trained-initiate |</span>
<span class="org-org-table">| 1998 |            10 |                10 |     1 | trained-initiate |</span>
<span class="org-org-table">| 1999 |            50 |                60 |     1 | trained-initiate |</span>
<span class="org-org-table">| 2000 |            50 |               110 |     1 | trained-initiate |</span>
<span class="org-org-table">| 2001 |           100 |               210 |     1 | trained-initiate |</span>
<span class="org-org-table">| 2002 |            60 |               270 |     1 | trained-initiate |</span>
<span class="org-org-table">| 2003 |           245 |               515 |     1 | trained-initiate |</span>
<span class="org-org-table">| 2004 |           115 |               630 |     1 | trained-initiate |</span>
<span class="org-org-table">| 2005 |           140 |               770 |     1 | trained-initiate |</span>
<span class="org-org-table">| 2006 |            60 |               830 |     1 | trained-initiate |</span>
<span class="org-org-table">| 2007 |           270 |              1100 |     2 | experienced      |</span>
<span class="org-org-table">| 2008 |           290 |              1390 |     2 | experienced      |</span>
<span class="org-org-table">| 2009 |           205 |              1595 |     2 | experienced      |</span>
<span class="org-org-table">| 2010 |           215 |              1810 |     2 | experienced      |</span>
<span class="org-org-table">| 2011 |           115 |              1925 |     2 | experienced      |</span>
<span class="org-org-table">| 2012 |           355 |              2280 |     3 | savvy            |</span>
<span class="org-org-table">| 2013 |           290 |              2570 |     3 | savvy            |</span>
<span class="org-org-table">| 2014 |           350 |              2920 |     3 | savvy            |</span>
<span class="org-org-table">| 2015 |            45 |              2965 |     3 | savvy            |</span>
<span class="org-org-meta-line">#+TBLFM: $2='(calc-eval (format "vsum(%s)" (vconcat (org-lookup-all $1 '(remote(accomplishments,@2$1..@&gt;$1)) '(remote(accomplishments,@2$3..@&gt;$3))))))::$3=vsum(@2$2..@+0$2)::$4='(org-lookup-last $3 '(remote(levels,@2$1..@&gt;$1)) '(remote(levels,@2$2..@&gt;$2)) '&gt;=);N::$5='(org-lookup-last $3 '(remote(levels,@2$1..@&gt;$1)) '(remote(levels,@2$3..@&gt;$3)) '&gt;=);L</span>
</pre>
</div>
<p>The <code>TBLFM</code> (table formula) line is very long, so let me break it down.</p>
<p><b>Points gained:</b></p>
<div class="org-src-container">
<pre class="src src-emacs-lisp"><span class="org-rainbow-delimiters-depth-1">(</span>calc-eval
 <span class="org-rainbow-delimiters-depth-2">(</span>format <span class="org-string">"vsum(%s)"</span>
         <span class="org-rainbow-delimiters-depth-3">(</span>vconcat
          <span class="org-rainbow-delimiters-depth-4">(</span>org-lookup-all
           $1
           '<span class="org-rainbow-delimiters-depth-5">(</span>remote<span class="org-rainbow-delimiters-depth-6">(</span>accomplishments,@2$1..@&gt;$1<span class="org-rainbow-delimiters-depth-6">)</span><span class="org-rainbow-delimiters-depth-5">)</span>
           '<span class="org-rainbow-delimiters-depth-5">(</span>remote<span class="org-rainbow-delimiters-depth-6">(</span>accomplishments,@2$3..@&gt;$3<span class="org-rainbow-delimiters-depth-6">)</span><span class="org-rainbow-delimiters-depth-5">)</span><span class="org-rainbow-delimiters-depth-4">)</span><span class="org-rainbow-delimiters-depth-3">)</span><span class="org-rainbow-delimiters-depth-2">)</span><span class="org-rainbow-delimiters-depth-1">)</span>
</pre>
</div>
<p>This uses <code>org-lookup-all</code> to look up the value of the first column (<code>$1</code>) in the <code>accomplishments</code> table, from the second row to the last row <code>@2..@&gt;</code>, looking in the first column (<code>$1</code>). It returns the values from the third column of the matching rows (<code>$3</code>). This is then passed through calc&#8217;s <code>vsum</code> function to calculate the sum.</p>
<p><b>Cumulative points:</b> <code>vsum(@2$2..@+0$2)</code> is the sum of the second column <code>$2</code> from the second row <code>@2</code> to the current row <code>@+0</code>.</p>
<p><b>Level:</b> This uses <code>org-lookup-last</code> to find the last value where the operator function returns true. In this case, testing the level from column <code>$3</code> against each of the values in the <code>levels</code> table&#8217;s column <code>$1</code> while the given level is greater than or equal to the value from <code>levels</code>. When it finds the last matching row, it returns the <code>$2</code> second column from it. <code>;N</code> means treat everything as a number.</p>
<p><code>org-lookup-first</code> is like <code>org-lookup-last</code>, but it returns the first matching row.</p>
<p><b>Adjective:</b> This one works like <b>Level</b> does, but it returns the value from column <code>$3</code> instead. I found that it converted the return values to <code>0</code> if I used <code>;N</code>, so I used <code>;L</code> instead.</p>
</div>
</div>
<div id="outline-container-unnumbered-3" class="outline-3">
<h3 id="unnumbered-3">Passing data to R or other external programs</h3>
<div id="text-unnumbered-3" class="outline-text-3">
<p>Of course, you&#8217;re not limited to things that Emacs can do. I wanted to summarize the data in graphs, so here&#8217;s what I did.</p>
<div class="org-src-container">
<pre class="src src-emacs-lisp">#+RESULTS: category_analysis

#+name: category_analysis
#+begin_src R <span class="org-builtin">:var</span> data=accomplishments <span class="org-builtin">:exports</span> both <span class="org-builtin">:results</span> graphics <span class="org-builtin">:file</span> quest_category.png <span class="org-builtin">:height</span> 300
library<span class="org-rainbow-delimiters-depth-1">(</span>plyr<span class="org-rainbow-delimiters-depth-1">)</span>
library<span class="org-rainbow-delimiters-depth-1">(</span>ggplot2<span class="org-rainbow-delimiters-depth-1">)</span>
categories &lt;- ddply<span class="org-rainbow-delimiters-depth-1">(</span>data, c<span class="org-rainbow-delimiters-depth-2">(</span><span class="org-string">"Category"</span><span class="org-rainbow-delimiters-depth-2">)</span>, summarize, Points=sum<span class="org-rainbow-delimiters-depth-2">(</span>XP<span class="org-rainbow-delimiters-depth-2">)</span><span class="org-rainbow-delimiters-depth-1">)</span>
cat_sorted &lt;- transform<span class="org-rainbow-delimiters-depth-1">(</span>categories, Category=reorder<span class="org-rainbow-delimiters-depth-2">(</span>Category, Points<span class="org-rainbow-delimiters-depth-2">)</span><span class="org-rainbow-delimiters-depth-1">)</span>
plot &lt;- ggplot<span class="org-rainbow-delimiters-depth-1">(</span>data = cat_sorted, aes<span class="org-rainbow-delimiters-depth-2">(</span>x = Category, y = Points<span class="org-rainbow-delimiters-depth-2">)</span><span class="org-rainbow-delimiters-depth-1">)</span>
plot &lt;- plot + geom_bar<span class="org-rainbow-delimiters-depth-1">(</span>stat=<span class="org-string">"identity"</span><span class="org-rainbow-delimiters-depth-1">)</span>
plot &lt;- plot + geom_text<span class="org-rainbow-delimiters-depth-1">(</span>aes<span class="org-rainbow-delimiters-depth-2">(</span>label = Points, x = Category, y = Points + 10, hjust=0<span class="org-rainbow-delimiters-depth-2">)</span><span class="org-rainbow-delimiters-depth-1">)</span>
plot &lt;- plot + scale_y_continuous<span class="org-rainbow-delimiters-depth-1">(</span>expand=c<span class="org-rainbow-delimiters-depth-2">(</span>0,70<span class="org-rainbow-delimiters-depth-2">)</span><span class="org-rainbow-delimiters-depth-1">)</span>
plot &lt;- plot + coord_flip<span class="org-rainbow-delimiters-depth-1">(</span><span class="org-rainbow-delimiters-depth-1">)</span>
print<span class="org-rainbow-delimiters-depth-1">(</span>plot<span class="org-rainbow-delimiters-depth-1">)</span>
#+end_src
</pre>
</div>
<p>I like including source code in published pages for fellow geeks, but having the results come first gives people more context for the source block. So I named the source block using the <code>#+name:</code> directive and defined a <code>#+RESULTS:</code> directive before it. The source block used the <code>:var</code> argument to bring the data in from the <code>accomplishments</code> table. With R blocks, the data becomes available as a data frame that you can then do interesting things with. I used the <code>:file</code> argument to save the output to <code>quest_category.png</code>.</p>
<p>Those are a few ways that you can get data out of Org Mode tables and into Emacs Lisp, other Org Mode tables, or external programs. As I learn more about Org Mode, I find myself using it for more of the things that I used to use Microsoft Excel for &#8211; tracking, analyzing, and even graphing. I found it a little difficult to piece together what I needed to do from the manuals and examples on the Web, so I hope this explanation will help you (and that it&#8217;ll help me when I forget the syntax, as I&#8217;m sure I will). If you come up with something really neat that uses Org Mode tables, tell me what you&#8217;ve figured out!</p>
</div>
</div>
<p>You can <a href="https://sachachua.com/blog/2015/01/getting-data-org-mode-tables/#comment">view 5 comments</a> or <a href="mailto:sacha@sachachua.com?subject=Comment%20on%20https%3A%2F%2Fsachachua.com%2Fblog%2F2015%2F01%2Fgetting-data-org-mode-tables%2F&body=Name%20you%20want%20to%20be%20credited%20by%20(if%20any)%3A%20%0AMessage%3A%20%0ACan%20I%20share%20your%20comment%20so%20other%20people%20can%20learn%20from%20it%3F%20Yes%2FNo%0A">e-mail me at sacha@sachachua.com</a>.</p>]]></content>
		</entry>
</feed>