Graham Enos

About

2026-04-29T00:00:00+00:00
While working as an Applied Research Mathematician for the US Department of Defense, I received my Ph.D. in Applied Mathematics from UNC Charlotte’s Math Department</a> in the Spring of 2013. After that, I worked as a Senior Data Scientist for Quantifind, Inc</a>. Then I was a Lead Research Scientist, Quantum Advantage for Rigetti Computing</a>, before starting my current role as Chief Scientist at Strangeworks</a>.
On this blog I explore those topics I never had the time to when I was working on my dissertation; essentially anything mathematical, computational, statistical, etc. that I happen to find interesting. For more about me, you can view my CV</a>, my GitHub</a>, my GitLab</a>, or my Mathstodon</a>.
Obligatory notice: my views do not necessarily reflect those of anyone kind enough to employ me.
(Unfortunately) obligatory notice: Nothing here was written with or by a large language model (LLM). Quoting the introduction of Modern Common Lisp with FSet</a>,
This document contains no LLM-generated text — zero, zip, nada. (Yes, I do use em-dashes and semicolons; yes, I have written every one myself.) </blockquote>
Tempest in a Teacup

2026-04-14T00:00:00+00:00
I shall laugh myself to death at this puppy-headed monster</h1>
Last year, Matt Keeter put out the Prospero challenge</a>. It’s a really great playground for learning about programming languages, compilation, constructive solid geometry</a>, and more!
I’m a bit late to the game, but in my “spare time,” I’ve been working on my own attempt. You can see the code here</a>.
So far, my approach has been pretty similar to others: after some canonicalization and a few simplifying/optimizing passes (e.g. eliding unused expressions), parallelize over disjoint chunks of the image, and use SIMD instructions to compute multiple pixels in a chunk at one time.
I’ve been tempted to push further, using intervals & quad-trees like in Matt Keeter’s very interesting talk</a>, but haven’t managed to pull it off yet. In the meantime, though, I’d like to highlight how important testing, especially property-based testing, has been.
Given the nature of the challenge, I’ve been leaning on some compiler wisdom</a> and reached for some snapshot testing. Since I’ve been working in Rust, I’m using `insta</code></a>. Being able to assert that my output matches the expected one byte-for-byte is pretty usefule.`
`Beyond that, I’ve also been using property-based testing. Instead of my usual go-toproptest</code></a>, I’ve been trying out the relatively new chaos_theory</code></a>. Besides being less macro-heavy, I think its design feels closer to hypothesis</code></a> from the Python world. Anecdotally, its tests ran faster for me, too! Having these testing tools have proven invaluable in approaching this challenge. I even found a bug in optimization code from one of the other entries!`



Parallel Means
2025-07-25T00:00:00+00:00
Don’t be mean</h1>
I got curious about parallelism—in Rust, but the same ideas probably apply
elsewhere—comparing using multiple CPUs via
rayon</code></a> and SIMD via
portable_simd</code></a>.
I suspected that which one would be the fastest would be highly dependent on
multiple factors, like the amount and distribution of the data involved and the
complexity of desired computation.</p>
To investigate, I wrote some code to calculate the mean of a collection of
32-bit floating point numbers in four different ways.1</a></sup>
The first is a sequential version of Welford’s online
algorithm</a>:</p>
pub fn</span> seq</span>(xs</span>: &</span>[</span>f32</span>])</span> -></span> f32</span> {</span></span>
    let</span> (</span>mut</span> n,</span> mut</span> mu)</span> =</span> (</span>0.0</span>,</span> 0.0</span>);</span></span>
    for</span> x</span> in</span> xs {</span></span>
        n</span> +=</span> 1.0</span>;</span></span>
        mu</span> +=</span> (x</span> -</span> mu)</span> /</span> n;</span></span>
    }</span></span>
    mu</span></span>
}</span></span></code></pre>
Next, we have a rayon</code>-ified version of Chan et. al.’s parallel
generalization</a>.
It first folds the same sequential version over each chunk in parallel, and
then merges the chunks together.2</a></sup></p>
pub fn</span> par</span>(xs</span>: &</span>[</span>f32</span>])</span> -></span> f32</span> {</span></span>
    xs</span>.</span>into_par_iter</span>()</span></span>
        .</span>fold</span>(</span></span>
            ||</span> (</span>0.0</span>,</span> 0.0</span>),</span></span>
            |</span>(n_old, mu_old), x</span>|</span> {</span></span>
                let</span> n</span> =</span> n_old</span> +</span> 1.0</span>;</span></span>
                let</span> mu</span> =</span> mu_old</span> +</span> (x</span> -</span> mu_old)</span> /</span> n;</span></span>
                (n, mu)</span></span>
            },</span></span>
        )</span></span>
        .</span>reduce</span>(</span></span>
            ||</span> (</span>0.0</span>,</span> 0.0</span>),</span></span>
            |</span>(n_a, mu_a), (n_b, mu_b)</span>|</span> {</span></span>
                let</span> n</span> =</span> n_a</span> +</span> n_b;</span></span>
                let</span> mu</span> =</span> mu_a</span> +</span> (mu_b</span> -</span> mu_a)</span> *</span> n_b</span> /</span> n;</span></span>
                (n, mu)</span></span>
            },</span></span>
        )</span></span>
        .</span>1</span></span>
}</span></span></code></pre>
I also developed two SIMD versions of the above, seq_simd<N></code> and
par_simd<N></code> for supported lane count N</code>.
These were relatively straightforward to write and test3</a></sup> based on their
non-SIMD counterparts.
The only parts that required a little more thought were collecting the N</code>
intermediate values and then “cleaning up” at the end; if the number of values
isn’t a multiple of the lane count N</code>, we need to pick up the last
(potentially up to N</code> - 1) elements.
I won’t include the SIMD code here, as I’ve already copy-pasted a fair amount
of code as it is, but it’s all available on my
GitHub</a>.</p>
Do you even bench?</h1>
Once these four different implementations were ready, I reached for
criterion</code></a> to try
and benchmark them.
I measured seq</code>, par</code>, seq_simd<N></code> and par_simd<N></code> for N</code> in \( \lbrace 8, 16,
32, 64\rbrace \).
For each power of ten between ten (\(10^1\)) and a billion (\(10^9\)), I
generated a vector of that length of f32</code>s uniformly from the unit interval,
and had criterion</code> measure the time it took each version to calculate the mean
of that vector.</p>
All the usual caveats about this type of benchmarking still apply!
I did the minimal due diligence—no other applications were running on my
laptop, e.g.—but your mileage may vary due to different hardware (in this
case, Apple M3 Max), data, computational load, etc.</p>
Results</h1>
After running the benchmarks, I collated and plotted the
results</a>.
Here’s the plot of #floats vs. nanoseconds (click on it for the full SVG); note
the log-log scale.</p>
</a></p>
To my eye, there’s a couple interesting things going on.
When dealing with a small number of floats, the sequential versions look
faster; indeed, the relative simplicity of seq</code> is the fastest for taking the
mean of ten values.
SIMD starts being useful pretty quickly, however; all four of the seq_simd<N></code>
versions are faster than seq</code> by the time we’re working with a thousand
floats.
Multi-CPU parallelism starts to be useful much later, but eventually wins out.
par</code> beats seq</code> by the time we reach \(10^5\), and each par_simd<N></code> beats
all four of the seq_simd<N></code>s by \(10^7\) and beyond.</p>
Fin.</h1>
All in all, the usual wisdom applies: benchmark for your particular use-case,
realistic workloads don’t look like benchmarks run in a vacuum, etc.
That said, this was a fun exercise!
The std::simd</code> APIs are rather pleasant to use; I look forward to them being
available on stable instead of only on nightly.
Please check out the code</a>
for more!</p>



I used f32</code> instead of f64</code> so as to fit more values in a single SIMD vector. ↩</a></p>
</li>

I smell a semigroup! ↩</a></p>
</li>

With both manual unit testing and
proptest</code></a> of course. ↩</a></p>
</li>
</ol>
</section>


APL Pi Day
2025-03-14T00:00:00+00:00
Happy \( \pi \) Day</h1>
In honor of today’s date</a>, I played
around with different ways of approximating everyone’s favorite constant in
Dyalog APL</a>.
What follows are translations of a few entries from
this</a> fun
Wikipedia list.
I tried to avoid anything that felt like cheating, e.g. using trigonometric
functions or other constants like \( \phi \).</p>
First, we set index-origin to its correct value ⎕io←0</code>.
Next up, here’s the familiar “throw darts at the unit square” Monte Carlo
approach (the first line sets our random seed):</p>
    ⎕rl</span>←</span>1729</span></span>
    6 1</span>⍴</span>{</span>4</span>×</span>(</span>+⌿÷≢</span>)</span>1</span>>+/×⍨?</span>⍵ 2</span>⍴</span>0</span>}</span>¨</span>10</span>*⍳</span>6</span></span>
4</span></span>
3.6</span></span>
3.12</span></span>
3.144</span></span>
3.162</span></span>
3.1458</span></span></code></pre>
Eschewing stochasticity, using Riemann sums for calculating the area of the
unit circle converges faster:</p>
      6 1</span>⍴</span>{(</span>4</span>÷×⍨</span>⍵</span>)</span>×+⌿</span>((</span>×⍨</span>⍵</span>)</span>-</span>(</span>×⍨</span>1</span>+⍳</span>⍵</span>))</span>*</span>0.5</span>}</span>¨</span>10</span>*⍳</span>6</span></span>
0</span></span>
2.904518326</span></span>
3.120417032</span></span>
3.139555467</span></span>
3.141391478</span></span>
3.141572616</span></span></code></pre>
as does this translation of the first entry in the iterative algorithm from the
Wikipedia list (as of this writing), viz.</p>
\[
a_0 = 1, \quad a_{n + 1} = \left(1 + \frac{1}{2n + 1}\right)a_n, \quad \pi = \lim_{n\to\infty}\frac{a_n^2}{n}
\]</p>
     6 1</span>⍴</span>{</span>⍵</span>÷⍨×⍨×⌿</span>{</span>1</span>+÷</span>1</span>+</span>2</span>×</span>⍵</span>}</span>⍳</span>⍵</span>}</span>¨</span>10</span>*⍳</span>6</span></span>
4</span></span>
3.221088997</span></span>
3.149456428</span></span>
3.14237815</span></span>
3.141671194</span></span>
3.141600508</span></span></code></pre>
Leibniz’s summation \(\sum_0^\infty \frac{(-1)^n}{2n+1}\) converges at a similar rate:</p>
      6 1</span>⍴</span>{</span>4</span>×+⌿</span>(</span>¯1</span>∘*÷</span>(</span>1</span>+</span>2</span>∘×</span>))</span>⍳</span>⍵</span>}</span>¨</span>10</span>*⍳</span>6</span></span>
4</span></span>
3.041839619</span></span>
3.131592904</span></span>
3.140592654</span></span>
3.141492654</span></span>
3.141582654</span></span></code></pre>
Though as the
corresponding</a>
entry notes, this is still sublinear.
The fastest one I tried is a slight reworking of</p>
\[
\sum_{n=0}^\infty \frac{(-1)^n}{4}\left(\frac{2}{4n+1}+\frac{2}{4n+2}+\frac{1}{4n+3}\right)
\]
This is the first entry from the Wikipedia’s list of infinite series which “are
efficient for calculating arbitrary binary digits of \(\pi\),” and credited
to the book Pi
Unleashed</a>.</p>
      15 1</span>⍴</span>{</span>+⌿</span>{((</span>2</span>÷</span>1</span>+</span>4</span>×</span>⍵</span>)</span>+</span>(</span>÷</span>1</span>+</span>2</span>×</span>⍵</span>)</span>+÷</span>3</span>+</span>4</span>×</span>⍵</span>)</span>÷</span>¯4</span>*</span>⍵</span>}</span>⍳</span>⍵</span>}</span>¨⍳</span>15</span></span>
0</span>          </span></span>
3.333333333</span></span>
3.114285714</span></span>
3.146356421</span></span>
3.140678766</span></span>
3.141777944</span></span>
3.141553701</span></span>
3.141601054</span></span>
3.141590807</span></span>
3.141593065</span></span>
3.141592561</span></span>
3.141592675</span></span>
3.141592649</span></span>
3.141592655</span></span>
3.141592653</span></span></code></pre>
This is a better approximation in 15 steps than the others reached in a million!</p>


Adding up uniform variates
2025-02-27T00:00:00+00:00
A wild e</em> appears</h1>
Here’s a fun math tidbit</a>:</p>

Select a random number between 0 and 1. Now select another and add it to the
first. Keep doing this, piling on random numbers. How many random numbers, on
average, do you need to make the total greater than 1? Answer: 2.71828….</p>
</blockquote>
— John Derbyshire, Prime
Obsession</a></em></p>
The inimitable MathWorld</a> has more
info on it, and the first link above has a nice proof sketch involving the
standard simplex (modified from Derbyshire’s book), as well as some Clojure code.</p>
Repo, Man</h1>
I’ve used this problem for a short programming exercise a number of times, and
I really like it.
Besides the fun math involved, it can give a decent understanding of how to get
things done in a language.
For instance, tackling it in Haskell can lead to thinking about monads (or
not), while doing it in a high performance language can make one think about
ways to squeeze out every last drop of performance.
I’ve collected implementations in a number of different languages
here</a>.
Some highlights and/or interesting points:</p>

The Rust version uses rayon</code> for parallelism and a
modification</a>
of Welford’s online algorithm for calculating the mean. This problem has a
lot of room for exploring parallelism, trading memory for time, and more.</li>
The array language versions in BQN and ngn/k do an approximation where
they build a large matrix of uniform variates and approximate from there,
rather than relying on loops or recursion.</li>
Writing WebAssembly by hand and calling it from TypeScript is surprisingly
painless (thanks to a pcg</code> implementation</a>
I found); what’s more, it’s competitive with the more “low-level” C/C++ and their ilk.</li>
Some things are much easier to package in a nix</code> flake than others.</li>
</ul>


Different Implementations of a Tiny Programming Language
2024-12-11T00:00:00+00:00
Random Art, Revisited</h1>
In a previous article</a>, we played around
with implementing a tiny programming language for generating random art.
In that article and the accompanying
code</a>, we implemented a trick
I learned from this
post</a>,
specifically matklad</code>’s</a> response.
In learning more about programming language implementation, I came across this
wonderful post</a> by
Professor Adrian Sampson, which examines the effects of flattening the abstract
syntax tree of a similar little language.
I thought it’d be interesting to perform a similar analysis with our random art
language implementation, so I put together a Rust
project</a> with four
different implementations.</p>
Implementations</h1>
The first implementation is the most basic, where we represent our AST directly:</p>
enum</span> Expr</span> {</span></span>
    X</span>,</span></span>
    Y</span>,</span></span>
    SinPi</span>(</span>Box</span><</span>Expr</span>>),</span></span>
    CosPi</span>(</span>Box</span><</span>Expr</span>>),</span></span>
    Mul</span>(</span>Box</span><</span>Expr</span>>,</span> Box</span><</span>Expr</span>>),</span></span>
    Avg</span>(</span>Box</span><</span>Expr</span>>,</span> Box</span><</span>Expr</span>>),</span></span>
    Thresh</span>(</span>Box</span><</span>Expr</span>>,</span> Box</span><</span>Expr</span>>,</span> Box</span><</span>Expr</span>>,</span> Box</span><</span>Expr</span>>),</span></span>
}</span></span></code></pre>
As mentioned last time</a>, it can be more
efficient store a single pointer (i.e. Box</code>), which my code confusingly calls
the “branch” version (apologies, but I wanted the names to be in alphabetical
order and it was the best I could come up with).
In this version, the main Expr</code> looks like</p>
enum</span> Expr</span> {</span></span>
    X</span>,</span></span>
    Y</span>,</span></span>
    SinPi</span>(</span>Box</span><</span>Expr</span>>),</span></span>
    CosPi</span>(</span>Box</span><</span>Expr</span>>),</span></span>
    Mul</span>(</span>Box</span><</span>Mul</span>>),</span></span>
    Avg</span>(</span>Box</span><</span>Avg</span>>),</span></span>
    Thresh</span>(</span>Box</span><</span>Thresh</span>>),</span></span>
}</span></span></code></pre>
where each of the intermediate structures looks like struct Mul(Expr, Expr)</code>, etc.</p>
After that, we move to a flat version similar to the one explored in Dr.
Sampson’s post</a>.
In this, we go back to the basic structure, wherein we store (potentially)
multiple things in a single Expr</code> variant:</p>
enum</span> Expr</span> {</span></span>
    X</span>,</span></span>
    Y</span>,</span></span>
    SinPi</span>(</span>ExprRef</span>),</span></span>
    CosPi</span>(</span>ExprRef</span>),</span></span>
    Mul</span>(</span>ExprRef</span>,</span> ExprRef</span>),</span></span>
    Avg</span>(</span>ExprRef</span>,</span> ExprRef</span>),</span></span>
    Thresh</span>(</span>ExprRef</span>,</span> ExprRef</span>,</span> ExprRef</span>,</span> ExprRef</span>),</span></span>
}</span></span></code></pre>
This time, though, the ExprRef</code>s are instead indices, 32 bit unsigned integers, like struct ExprRef(u32)</code>.
We then use a pool of expressions, struct ExprPool(Vec<Expr>)</code>; during its
construction, every ExprRef</code> index points to a valid index into this vector.
This controls memory layout even more tightly than our branch version, though
interpretation still
uses</a>
an additional vector of the same length.</p>
The final version follows this
comment</a>
by Bob Nystrom</a>, modifying our flat version
to implement a stack-based bytecode interpreter.
Every subexpression is interpreted before the larger expression that uses it,
and the ordering of items is all that matters.
We can eschew any kind of pointing or indexing, and move directly to a stack of individual instructions:</p>
enum</span> Expr</span> {</span></span>
    X</span>,</span></span>
    Y</span>,</span></span>
    SinPi</span>,</span></span>
    CosPi</span>,</span></span>
    Mul</span>,</span></span>
    Avg</span>,</span></span>
    Thresh</span>,</span></span>
}</span></span></code></pre>
These are then stored in order in a struct ExprPool(Vec<Expr>)</code> during
construction, and
interpretation</a>
uses a single stack.</p>
Do you even bench(mark)?</h1>
For benchmarking, for each version we build up a random expression of “depth”
26 in the same fashion as last post, then evaluate that expression with a
pseudorandomly chosen x</code> and y</code>.
Reaching for the amazing hyperfine</code></a> to
time our various approaches:</p>
</p>
We get the following times:</p>
</p>
Interestingly, there’s a large time improvement in moving from the basic to the branch version alone!
From there, we get further improvements moving to the flat and stack versions,
though the savings are less dramatic.</p>
To keep myself honest, I also double-checked that versions yield equivalent
results (at least, up to a certain depth) via some property-based
tests</a>
with the help of the exceptional proptest</code>
crate</a>.</p>
Coda & Code</h1>
As always, it was fun and interesting to take the lessons learned in reading and see their impacts in practice.
Please feel free to check out the code</a> for more details!</p>


Here, Have a Fractal
2024-11-06T00:00:00+00:00
Not feeling it</h1>
Today isn’t great.
I don’t like it.
So here, to take minds off things, have a fractal.</p>
</p>
Ugly Rust code, using image</code> and num_complex</code>:</p>
use</span> image</span>::</span>{</span>ImageBuffer</span>,</span> Rgb</span>};</span></span>
use</span> num_complex</span>::</span>Complex</span>;</span></span>
</span>
fn</span> main</span>() {</span></span>
    let</span> img</span> =</span> 3200</span>;</span></span>
    let</span> scale</span> =</span> 3.0</span> /</span> img</span> as</span> f32</span>;</span></span>
    let</span> max_iter</span> =</span> 42.0</span>;</span></span>
    let</span> imgbuf</span> =</span> ImageBuffer</span>::</span>from_par_fn</span>(img, img,</span> |</span>x, y</span>|</span> {</span></span>
        let</span> c</span> =</span> Complex</span>::</span>new</span>(x</span> as</span> f32</span> *</span> scale</span> -</span> 2.25</span>, y</span> as</span> f32</span> *</span> scale</span> -</span> 1.5</span>);</span></span>
        let</span> (</span>mut</span> z,</span> mut</span> i)</span> =</span> (</span>Complex</span>::</span><</span>f32</span>></span>::</span>ZERO</span>,</span> 0.0</span>);</span></span>
        while</span> i</span> <</span> max_iter</span> &&</span> z</span>.</span>norm_sqr</span>()</span> <=</span> 16.0</span> {</span></span>
            z</span> =</span> z</span> *</span> z</span> +</span> c;</span></span>
            i</span> +=</span> 1.0</span>;</span></span>
        }</span></span>
        let</span> n</span> =</span> i</span> +</span> 1.0</span> -</span> (</span>0.5</span> *</span> z</span>.</span>norm</span>()</span>.</span>ln</span>())</span>.</span>log2</span>();</span></span>
        let</span> h</span> =</span> (n</span> *</span> 360.0</span> /</span> max_iter</span> +</span> 210.0</span>)</span> %</span> 360.0</span>;</span></span>
        let</span> v</span> =</span> (i</span> <</span> max_iter)</span> as</span> u8</span> as</span> f32</span>;</span></span>
        let</span> f</span> = |</span>n</span>:</span> f32</span>|</span> {</span></span>
            let</span> k</span> =</span> (n</span> +</span> (h</span> /</span> 60.0</span>))</span> %</span> 6.0</span>;</span></span>
            (</span>255.0</span> *</span> (v</span> -</span> 0.5</span> *</span> v</span> *</span> 0</span>f32</span>.</span>max</span>(k</span>.</span>min</span>(</span>4.0</span> -</span> k)</span>.</span>min</span>(</span>1.0</span>))))</span> as</span> u8</span></span>
        };</span></span>
        Rgb</span>([</span>f</span>(</span>5.0</span>),</span> f</span>(</span>3.0</span>),</span> f</span>(</span>1.0</span>)])</span></span>
    });</span></span>
    imgbuf</span>.</span>save</span>(</span>"mandelbrot.png"</span>)</span>.</span>unwrap</span>();</span></span>
}</span></span></code></pre>


Tropical Semiring in Dyalog APL
2024-04-04T00:00:00+00:00
A few of my favorite things</h1>
The Tropical (min-plus)
semiring</a> is one of my
favorite examples of how changing one’s perspective can make difficult problems
much simpler.1</a></sup>
In this semiring, instead of using the usual addition and multiplication,
we replace them with minimum and addition, so, for instance, \( 1 \oplus 2 = 1
\) and \( 3 \otimes 2 = 5 \).
I’ve written</a> and
presented</a> about
this</a> before.</p>
Links from the farm</h1>
Recently, someone on the APL Farm</a> posted
an excellent
article</a>
by sigfpe</code> (AKA Dan</a>
Piponi</a>), which prompted me to post a
different</a>
article where they work with the min-+ semiring.
I also posted this article</a> by
Russell O’Connor, which is one of my all-time favorites.
In it, there’s an example where the distances between nodes in a graph gets
turned into a Tropical matrix, and algorithms for computing shortest distances
become iterated linear algebra over this different semiring.
And then I wanted to work in APL instead of Haskell…</p>
An open dialogue</h1>
After a bit of head scratching, I realized that Dyalog
APL</a> is really good for manipulating matrices over the
min-+ semiring.
Taking exampleGraph2</code> from O’Connor’s blog post, and letting \( 10^{20} \)
stand in for \( \infty \)2</a></sup>:</p>
inf</span>←</span>1e20</span></span>
dist</span>←</span>[</span>7 9</span> inf inf</span> 14 7</span></span>
      0 10 15</span> inf inf</span></span>
      9 10  0 11</span> inf</span> 2</span></span>
      inf</span> 15 11 0 6</span> inf</span></span>
      inf inf inf</span> 6 0 9</span></span>
      14</span> inf</span> 2</span> inf</span> 9 0</span>]</span></span></code></pre>
we have the following distance matrix</p>
      dist</span></span>
0.0E0  7.0E0  9.0E0  1.0E20 1E20 1.4E1</span></span>
7.0E0  0.0E0  1.0E1  1.5E1  1E20 1.0E20</span></span>
9.0E0  1.0E1  0.0E0  1.1E1  1E20 2.0E0</span></span>
1.0E20 1.5E1  1.1E1  0.0E0  6E0  1.0E20</span></span>
1.0E20 1.0E20 1.0E20 6.0E0  0E0  9.0E0</span></span>
1.4E1  1.0E20 2.0E0  1.0E20 9E0  0.0E0</span></span></code></pre>
With this matrix, representing distances between vertices in our graph, we can
use the APL matrix product operator .</code> with different operations</em> to perform
our calculations; instead of +</code> and ×</code> for addition and multiplication, we
turn to ⌊</code> (min) and +</code>; then dist ⌊.+ dist</code> gives us the two-hop distances:</p>
      dist</span> ⌊.+</span> dist</span></span>
 0  7  9 20 23 11</span></span>
 7  0 10 15 21 12</span></span>
 9 10  0 11 11  2</span></span>
20 15 11  0  6 13</span></span>
23 21 11  6  0  9</span></span>
11 12  2 13  9  0</span></span></code></pre>
We can make this more succinct with ⍨</code>, telling the interpreter to apply our
function with dist</code> as both arguments:</p>
      ⌊.+⍨</span> dist</span></span>
 0  7  9 20 23 11</span></span>
 7  0 10 15 21 12</span></span>
 9 10  0 11 11  2</span></span>
20 15 11  0  6 13</span></span>
23 21 11  6  0  9</span></span>
11 12  2 13  9  0</span></span></code></pre>
I was very excited that I could write down the steps of the
Gauss-Jordan-Floyd-Warshall-Kleene algorithm in so few characters!
Moreover, iterating that step until convergence is similarly succinct using the
power operator ⍣</code>; we can keep running ⌊.+</code> until the output matches (≡</code>) the input:</p>
      ⌊.+⍨⍣≡</span>dist</span></span>
 0  7  9 20 20 11</span></span>
 7  0 10 15 21 12</span></span>
 9 10  0 11 11  2</span></span>
20 15 11  0  6 13</span></span>
20 21 11  6  0  9</span></span>
11 12  2 13  9  0</span></span></code></pre>
Imagine my surprise when I searched for “distance” in Iverson’s notation as a
tool of thought</a>:</p>

and if p</code> gives distances from a source to transhipment points and q</code> gives
distances from the transhipment points to the destination, then p⌊.+q</code> gives
the minimum distance possible.</p>
</blockquote>



As Alan Kay says</a>, point
of view is worth 80 IQ points. ↩</a></p>
</li>

I tried this in BQN</a> as well, since it
has \( \infty \), but the linear algebra and fixed point iteration are
nicer in APL. ↩</a></p>
</li>
</ol>
</section>


Evaluating quantum generative models via imbalanced data classification benchmarks
2023-08-22T00:00:00+00:00
New paper, who dis?</h1>
I’m excited to announce that our paper on using classification performance on
imbalanced datasets as a proxy for measuring generative model quality up on
the arXiv</a>!
It combines a lot of interesting techniques, like post-hoc testing, the
Bayesian boostrap, and explainable AI techniques, all to get an idea of how
well a quantum generative model performs.
Please check it out!</p>


A Rusty Quantum Interpreter
2023-08-14T00:00:00+00:00
The Original</h1>
Robert Smith, a.k.a. stylewarning</code>, has a lovely blog
post</a> that walks
through implementing an interpreter for a “general-purpose quantum programming
language called \( \mathscr{L} \).”
In only 150 lines of Common Lisp, the
implementation</a> is
featureful, self-contained, and a delight to read.</p>
Imitation is the highest form of flattery</h1>
At first I was content with only reading the post and code, but then I
saw</a> someone else
had put together an OCaml implementation as well.
The game was afoot!</p>
Carcinization</h1>
Eventually I put together a Rust
implementation</a>.
This version weights in at more than twice the line count as the
original—even though it relies on
faer</code></a> for linear
algebra instead of implementing it by hand!1</a></sup>
However, it does have a couple of features not present in the original (or
OCaml) implementation(s):</p>

constructing a Machine</code> takes an unsigned 64-bit seed</code> for repeatable PRNG
behavior; and</li>
with the help of the peg</code>
crate</a>, I added parsing, so one can pass
"H 0\nMEASURE"</code> as a string and get an interpretable quantum program.</li>
</ul>
Code or it didn’t happen</h1>
The implementation is
here</a> in my
catch-all “workbench” repository.
It’s definitely not</em> production-worthy code; it’s got an assert!</code> that will
panic if not met, and it uses expect</code> to pave over some errors.2</a></sup>
Please feel free to take a look and let me know what you think!</p>



The original version used ndarray</code>, since I was more familiar with that crate. ↩</a></p>
</li>

Update 2024-06-27: As of this PR</a>, the code is a little more respectable now, with proper error handling via Result</code> and no panics; that said, I’d still advise against using it in a production setting. ↩</a></p>
</li>
</ol>
</section>


Random Art with a Tiny Language
2023-06-02T00:00:00+00:00
PL Resources Everywhere</h1>
In playing around with OCaml</a>, I’ve
spent some spare time perusing more programming language resources.
There are a bunch out there, especially in this era of online learning; for
instance, Cornell has made some
great</a>
resources</a>
available.
In looking specifically for more ML-ish1</a></sup> flavored ones, I came across
this</a> fun undergrad
homework set.
In the second problem, it asks you to implement a tiny (recursively
defined) language of expressions encoding functions from the unit square to the
unit interval, \( [0, 1]^2 \mapsto [0, 1] \).</p>
type</span> expr</span> =</span> VarX</span> </span></span>
          |</span> VarY</span> </span></span>
          |</span> Sine</span> of expr </span></span>
          |</span> Cosine</span> of expr </span></span>
          |</span> Average</span> of expr</span> *</span> expr </span></span>
          |</span> Times</span> of expr</span> *</span> expr </span></span>
          |</span> Thresh</span> of expr</span> *</span> expr</span> *</span> expr</span> *</span> expr </span></span></code></pre>
After you learn to build them up randomly, you can evaluate them across the
unit square to generate pictures—have each output value encode either a
greyscale value, or one of an RGB triple.
Note to undergrads: please</em> don’t cheat off me; you’ll only rob yourself of a
fun learning experience.</p>
No Peeking!</h1>
This was a fun exercise in OCaml, but I also found it instructive to tackle it
in Rust</a> and compare.
Besides the superficial differences, Rust’s concern with and ability to control
memory usage is apparent.
I defined the core enumeration thusly:</p>
pub enum</span> Expr</span> {</span></span>
    X</span>,</span></span>
    Y</span>,</span></span>
    SinPi</span>(</span>Box</span><</span>Expr</span>>),</span></span>
    CosPi</span>(</span>Box</span><</span>Expr</span>>),</span></span>
    Mul</span>(</span>Box</span><</span>Mul</span>>),</span></span>
    Avg</span>(</span>Box</span><</span>Avg</span>>),</span></span>
    Thresh</span>(</span>Box</span><</span>Thresh</span>>),</span></span>
}</span></span></code></pre>
I renamed some variants to more accurately express their intent; e.g.
SinPi(expr)</code> evaluates to \( sin(\pi \cdot \) expr</code> \(\!)\).
More importantly, I reworked the recursive variants to each contain (at most) a
single Box</code> (i.e. unique pointer to a heap allocated value); for instance,
Expr::Mul</code> contains a Box<Mul></code>, where Mul</code> is defined as</p>
pub struct</span> Mul</span> {</span></span>
    e1</span>:</span> Expr</span>,</span></span>
    e2</span>:</span> Expr</span>,</span></span>
}</span></span></code></pre>
I took up this trick after
learning</a>
more about it from the inimitable matklad</code></a>.
As the linked response says, this keeps the size of the base enum smaller.
In the source code, I’ve got a property
test</a>
that asserts the size of an arbitrary2</a></sup> Expr</code> is exactly 16 bytes3</a></sup>:</p>
#[test]</span></span>
fn</span> size_of</span>(e</span> in</span> arb_expr</span>()) {</span></span>
    assert_eq!</span>(</span>size_of_val</span>(</span>&</span>e),</span> 16</span>)</span></span>
}</span></span></code></pre>
If we instead went with a more direct translation of the OCaml code, like</p>
diff --git a/src/lib.rs b/src/lib.rs</span></span>
index 3b04a74..68c600c 100644</span></span>
--- a/src/lib.rs</span></span>
+++ b/src/lib.rs</span></span>
@@ -7,9 +7,9 @@</span> pub enum Expr {</span></span>
     Y,</span></span>
     SinPi(Box<Expr>),</span></span>
     CosPi(Box<Expr>),</span></span>
-    Mul(Box<Mul>),</span></span>
-    Avg(Box<Avg>),</span></span>
-    Thresh(Box<Thresh>),</span></span>
+    Mul { e1: Box<Expr>, e2: Box<Expr> },</span></span>
+    Avg { e1: Box<Expr>, e2: Box<Expr> },</span></span>
+    Thresh { e1: Box<Expr, e2: Box<Expr>, e3: Box<Expr>, e4: Box<Expr> },</span></span>
 }</span></span></code></pre>
then this property test would no longer pass, as the enum needs to be large
enough to contain its largest variant.</p>
Pretty Pics Plz</h1>
Once I coded up the library, I also put together a small
executable</a> to
take in some parameters and generate greyscale or RGB pictures, similar to the
original exercise.
With that defined, I could run it a bunch of times using
jot</code></a> to
generate random seeds and parallel</code></a>
to use multiple cores:</p>
jot</span> -</span>w</span> %</span>i</span> -</span>r</span> 20 0 1000000000</span> |</span> parallel</span> --</span>bar</span> ./</span>target</span>/</span>release</span>/</span>ra</span> -</span>s {}</span> -</span>d</span> 7</span> -</span>r</span></span></code></pre>
Here are some gems I found combing through the PNGs:</p>

    
        </td>
        </td>
        </td>
    </tr>
    

        </td>
        </td>
        </td>
    </tr>
    

        </td>
        </td>
        </td>
    </tr>
</table>



ML</a> as in OCaml
and SML, not machine learning. ↩</a></p>
</li>

I really</em> like property-based testing, in this case with
proptest</code></a>. ↩</a></p>
</li>

Since
size_of_val</code></a>
returns the size of the pointed-to value in bytes, this test returns 16 on
my 64-bit laptop; it will probably fail on a 32-bit architecture. Be sure
to check out
this</a> awesome
article for more on Rust enums. ↩</a></p>
</li>
</ol>
</section>


Kompreni: Understanding OCaml with Abstract Algebra Examples
2023-05-11T00:00:00+00:00
Why and wherefore</h1>
I’ve wanted to level up my OCaml</a> understanding for a
while now, but between doing a fair amount of work in Rust at my current job
and already having a bit of Haskell, I wasn’t sure I could justify the
“distraction.”
But between some excellent</a>
resources available online</a>, some
interesting reading (more on that in a moment), and the newest 5.0.0
release</a>, I decided it was time to dig in!</p>
Oleg Kiselyov does something cool, news at 11</h1>
The first reading topic I’ve stumbled across recently come from an extremely
prolific CS researcher, Oleg Kiselyov</a> who has written
a lot of amazing papers and code.
One such project is the tagless-final</a>
style of embedding domain-specific languages in a typed functional language
like Haskell or OCaml.
I can’t do it justice, it’s truly amazing stuff, you should go read it
yourself.
But!
Having seen some of the work before in Haskell, it was intriguing to look
through this work and compare and contrast Haskell and OCaml versions.</p>
Modules modules modules</h1>
The second topic is modules, as in “module systems” and “modular
programming”</a>.1</a></sup>
They’re really interesting; indeed, some say modules matter
most</a>.
OCaml (like SML and other related languages) allow for some really interesting
flexibility around designing and implementing interfaces.
I won’t be able to explain as well as the above link or
this</a>
more verbose and less esoteric explanation.</p>
Algebraic building blocks</h1>
Which leads me to the point of my little
project named kompreni</code></a>.
While it is certainly nowhere near as complete as, say,
bastet</code></a>,2</a></sup> I wanted to use
describing abstract algebra ideas as a way to explore expressing things in
OCaml.</p>
We’ll start with a semigroup</a>; a
semigroup consists of a set \( S \) and an associative binary operation \( \cdot \)
on that set; that is, for any \(x\), \(y\), and \(z\) in \(S\), it must be the
case that \( x \cdot (y \cdot z) = (x \cdot y) \cdot z \).</p>
In kompreni</code>, I’ve written a Signature</code> for semigroups:</p>
module</span> type</span> Signature =</span> sig</span></span>
  </span>type</span> t</span></span>
  </span>val</span> (</span> +&</span> )</span> :</span> t</span> -></span> t</span> -></span> t</span></span>
end</span></span></code></pre>
along with an accompanying Laws</code> functor, a function which takes any</em> module
implementing the above Signature</code> and creates a new module:</p>
module</span> Laws</span> (</span>S</span> :</span> Signature</span>)</span> =</span> struct</span></span>
  open</span> S</span></span>
  </span>let</span> associative</span> </span>x</span> </span>y</span> </span>z</span> =</span> x</span> +&</span> (y</span> +&</span> z)</span> =</span> x</span> +&</span> y</span> +&</span> z</span></span>
end</span></span></code></pre>
One thing I miss compared to Haskell is being able to specify the precedence
and associativity of user-defined operators like +&</code>.
Since it begins with the plus symbol, OCaml treats it like a plus; ah well.
That means that this expression above, when formatted with dune fmt</code></a>, looks like it’s missing some parentheses;
OCaml says +</code> is left-associative, so it says +&</code> is too.</p>
With that signature and functor together, I’ve next turned to
qcheck</code></a> for property-based testing in
OCaml.
For instance, in kompreni</code>’s test suite I’ve got the following.
First we define a base signature for testing:</p>
module</span> type</span> Testable =</span> sig</span></span>
  </span>type</span> t</span></span>
  </span>val</span> gen</span> :</span> t</span> QCheck2</span>.</span>Gen</span>.</span>t</span></span>
end</span></span></code></pre>
Then we combine it with the Signature</code> above to actually test the associative</code> property:</p>
module</span> SemigroupLaws</span> (</span>X</span> :</span> Testable</span>) (</span>S</span> :</span> Semigroup</span>.</span>Signature </span>with</span> type</span> t</span> = X</span>.</span>t)</span> =</span></span>
struct</span></span>
  include</span> Semigroup</span>.</span>Laws</span> (</span>S</span>)</span></span>
  </span>let</span> tests</span> =</span></span>
    [</span></span>
      make_test</span></span>
        QCheck2</span>.</span>Gen</span>.</span>(triple</span> X</span>.</span>gen</span> X</span>.</span>gen</span> X</span>.</span>gen)</span></span>
        "associative"</span> (uncurry3 associative)</span>;</span></span>
    ]</span></span>
end</span></span></code></pre>
With other scaffolding (like uncurry3</code>) defined elsewhere, this takes two
modules—one implementing the testable interface and another that is a
Semigroup</code> with the same</em> internal type t</code>—and defines a list of qcheck</code> tests.</p>
kompreni</code> also contains monoids</a>,
semigroups with an identity element we’ll call zero</code>:</p>
module</span> type</span> Signature =</span> sig</span></span>
  include</span> Semigroup</span>.</span>Signature</span></span>
  </span>val</span> zero</span> :</span> t</span></span>
end</span></span></code></pre>
Note that include</code> statement, which says “bring in the body of the Semigroup</code>
signature, too.”
We also have more Laws</code> for monoids to fulfill, in addition to associative</code>:</p>
module</span> Laws</span> (</span>M</span> :</span> Signature</span>)</span> =</span> struct</span></span>
  open</span> M</span></span>
  include</span> Semigroup</span>.</span>Laws</span> (</span>M</span>)</span></span>
  </span>let</span> left_id</span> </span>x</span> =</span> zero</span> +&</span> x</span> =</span> x</span></span>
  </span>let</span> right_id</span> </span>x</span> =</span> x</span> +&</span> zero</span> =</span> x</span></span>
end</span></span></code></pre>
which we can also test:</p>
module</span> MonoidLaws</span> (</span>X</span> :</span> Testable</span>) (</span>M</span> :</span> Monoid</span>.</span>Signature </span>with</span> type</span> t</span> = X</span>.</span>t)</span> =</span></span>
struct</span></span>
  include</span> Monoid</span>.</span>Laws</span> (</span>M</span>)</span></span>
  </span>let</span> tests</span> =</span></span>
    </span>let</span> module</span> SL</span> = SemigroupLaws</span> (</span>X</span>) (</span>M</span>)</span> </span>in</span></span>
    List</span>.</span>map (uncurry2 (make_test</span> X</span>.</span>gen))</span></span>
      [</span> (</span>"left id",</span> left_id)</span>;</span> (</span>"right id",</span> right_id)</span> ]</span></span>
    @</span> SL</span>.</span>tests</span></span>
end</span></span></code></pre>Semirings in paradise</h1>
This post is already getting too long, so I’ll finish with my favorite example.
I think this setup shows the power of OCaml modules, but also shows a place
where either I don’t know how to use them well enough (very</em> plausible!) or
they fall just a tad short of what I’d like.</p>
Consider semirings</a>, which consist of
a set of elements \( S \), two</em> operations \( + \) and \( \cdot \), and
two special elements \(0\) and \(1\) such that:</p>

\((S, +, 0)\) forms a commutative (i.e. \(x + y = y + x\)) monoid,</li>
\((S, \cdot, 1)\) is a monoid,</li>
multiplication of anything by \(0\) returns \(0\), and</li>
multiplication \(\cdot\) distributes over \(+\).</li>
</ul>
My absolute favorite semiring is the Tropical
semiring</a>, also known as the
min-plus semiring, which consists of:</p>

\( \mathbb{R} \cup \infty \), numbers that are either finite real numbers or infinite,</li>
\(x + y\) is defined to be \(\min(x, y)\) wiht identity element \(\infty\), and</li>
\(x \cdot y\) is defined to be \(x + y\) in the usual sense, so its identity is zero.</li>
</ul>
Here’s kompreni</code>’s implementation3</a></sup>:</p>
module</span> MinPlus</span> =</span> struct</span></span>
  </span>type</span> t</span> =</span> Finite</span> of</span> Q</span>.</span>t</span> |</span> Infinite</span></span>
  </span>let</span> (</span> +&</span> )</span> </span>a</span> </span>b</span> =</span></span>
    match</span> (a</span>,</span> b)</span> with</span></span>
    |</span> Finite</span> </span>x</span>,</span> Finite</span> </span>y</span> -></span> Finite</span> (</span>Q</span>.</span>max x y)</span></span>
    |</span> Finite</span> _</span>,</span> _</span> -></span> a</span></span>
    |</span> _</span>,</span> Finite</span> _</span> -></span> b</span></span>
    |</span> _</span>,</span> _</span> -></span> Infinite</span></span>
  </span>let</span> zero</span> =</span> Infinite</span></span>
  </span>let</span> (</span> *&</span> )</span> </span>a</span> </span>b</span> =</span></span>
    match</span> (a</span>,</span> b)</span> with</span></span>
    |</span> Finite</span> </span>x</span>,</span> Finite</span> </span>y</span> -></span> Finite</span> (</span>Q</span>.</span>add x y)</span></span>
    |</span> _</span>,</span> _</span> -></span> Infinite</span></span>
  </span>let</span> one</span> =</span> Finite</span> Q</span>.</span>zero</span></span>
end</span></span></code></pre>
And here are the laws that a Semiring</code> must abide by in kompreni</code>:</p>
module</span> Laws</span> (</span>S</span> :</span> Signature</span>)</span> =</span> struct</span></span>
  open</span> S</span></span>
  include</span> Commutative_monoid</span>.</span>Laws</span> (</span>S</span>)</span></span>
  (* (S, 1, *&) is also a Monoid *)</span></span>
  </span>let</span> times_associative</span> </span>x</span> </span>y</span> </span>z</span> =</span> x</span> *&</span> (y</span> *&</span> z)</span> =</span> x</span> *&</span> y</span> *&</span> z</span></span>
  </span>let</span> times_left_one</span> </span>x</span> =</span> one</span> *&</span> x</span> =</span> x</span></span>
  </span>let</span> times_right_one</span> </span>x</span> =</span> x</span> *&</span> one</span> =</span> x</span></span>
  (* Zero annihilates *)</span></span>
  </span>let</span> zero_annihilates_left</span> </span>x</span> =</span> zero</span> *&</span> x</span> =</span> zero</span></span>
  </span>let</span> zero_annihilates_right</span> </span>x</span> =</span> x</span> *&</span> zero</span> =</span> zero</span></span>
  (* Distributivity *)</span></span>
  </span>let</span> left_distributive</span> </span>x</span> </span>y</span> </span>z</span> =</span> x</span> *&</span> (y</span> +&</span> z)</span> =</span> (x</span> *&</span> y)</span> +&</span> (x</span> *&</span> z)</span></span>
  </span>let</span> right_distributive</span> </span>x</span> </span>y</span> </span>z</span> =</span> (x</span> +&</span> y)</span> *&</span> z</span> =</span> (x</span> *&</span> z)</span> +&</span> (y</span> *&</span> z)</span></span>
end</span></span></code></pre>
I really</em> like being able to include previous law definitions in new ones; for
instance, having defined the Commutative_monoid.Laws</code> functor, the line
include Commutative_monoid.Laws (S)</code> ensures that I include all those laws
into my tests for almost free—ensuring that I test \((S, +, 0)\) is indeed
a commutative monoid.
However, I don’t see a way to cleverly check \((S, ⋅, 1)\) is also a monoid
without writing out the times_</code> rules explicitly.
Maybe I need more OCaml module & functor goodness?
bastet</code> forgoes wrapping the monoid definitions and laws into the semiring one, for what it’s worth.</p>
Fin.</h1>
While I miss the usability of cargo</code> from Rust and some fun things from
Haskell, OCaml is fun and eye-opening!
Please feel free to head to GitHub</a> to
check out the code.</p>



Not to be confused with a module over a ring</a> or one of the many other</a> definitions of “module.” ↩</a></p>
</li>

Nor will it likely ever</em> be. ↩</a></p>
</li>

Rather than floats, this uses rational numbers from zarith</code></a> for finite values. Using \(\mathbb{Q}\) is not the same as using \(\mathbb{R}\), but this still forms a semiring! Also it’s really, really</em> hard to represent arbitrary real numbers in software. For instance, the associativity tests fail if you use OCaml floats, because floating point numbers aren’t real</a>. ↩</a></p>
</li>
</ol>
</section>


Scum: write me a Scheme-ish
2023-04-20T00:00:00+00:00
A wretched hive of scum and villainy</h1>
With it being my birthday today, I took a little bit of time to noodle on a pet project.
For all the computer science classes I’ve taken, books I’ve read, course notes I’ve looked through, and code I’ve written, reviewed, or perused, I had yet to implement any sort of Lisp-like language myself.
Having decided to change that, I’ve been toying around for the last month or so on scum</code></a>, a Scheme-ish, Lisp-like language implementation.
It’s by no means good1</a></sup> in any sense of the word; it’s incomplete, under-commented, probably loaded with bugs, undoubtedly slow, and profligate with its memory usage.
However, as of today, it’s cleared the bar I set for myself for being a reasonable-ish implementation: I can define and call lambdas</a> from the REPL!</p>
How long is a piece of string</h1>
There’s not a lot of code currently, so feel free to peruse the repo</a> yourself:</p>
~</span>/github/scum ∃ l</span></span>
Permissions</span> Size User  Date Modified Git Name</span></span>
.rw-r--r--@</span>  25k genos</span> 13</span> Apr 14:58</span>   --</span> Cargo.lock</span></span>
.rw-r--r--@</span>   94</span> genos</span> 13</span> Apr 14:58</span>   --</span> Cargo.toml</span></span>
.rw-r--r--@</span> 1.1k genos</span> 13</span> Apr 14:58</span>   --</span> LICENSE</span></span>
.rw-r--r--@</span>   56</span> genos</span> 13</span> Apr 14:58</span>   --</span> README.md</span></span>
drwxr-xr-x@</span>    - genos</span> 20</span> Apr 15:18</span>   --</span> scum-lib</span></span>
drwxr-xr-x@</span>    - genos</span> 13</span> Apr 14:58</span>   --</span> scum-repl</span></span>
drwxr-xr-x@</span>    - genos</span> 20</span> Apr 13:22</span>   -I</span> target</span></span>
~</span>/github/scum ∃ tokei</span></span>
===============================================================================</span></span>
 Language</span>            Files        Lines         Code     Comments       Blanks</span></span>
===============================================================================</span></span>
 Markdown</span>                1            3            0            2            1</span></span>
 Pest</span>                    1           38           26            8            4</span></span>
 Rust</span>                    8          628          577            9           42</span></span>
 TOML</span>                    3           27           23            0            4</span></span>
===============================================================================</span></span>
 Total</span>                  13          696          626           19           51</span></span>
===============================================================================</span></span></code></pre>
By no means as succinct as, say tinylisp</code></a>, nor as complete as the mal</code> implementations</a>, scum</code> consists of</p>

a Rust library which defines the basic expression type, as well as a single step of the read-eval-print loop, and</li>
a Rust executable which does little more than reach out to rustyline</code></a> to provide an interactive REPL:</li>
</ul>
λ>  (</span>define</span> square (</span>lambda</span> (</span>x</span>) (</span>*</span> x x)))</span></span>
(</span>lambda</span> (</span>x</span>) (</span>*</span> x x))</span></span>
λ>  (square </span>39</span>)</span></span>
1521</span></span></code></pre>
I originally started in Haskell, but decided to shift to Rust because I kind of wanted this to be an exercise in satisfying an annoying pedant; Haskell provides so much already, it almost felt like cheating to use it.
Besides the aforementioned rustyline</code> crate in scum-repl</code>, I’ve used pest</code> and pest_derive</code></a> for parsing and thiserror</code></a> for organizing the sundry reasons things can go wrong.
While pest</code> was new to me and has been fun, I’ve used thiserror</code> on other projects at $WORK</code> before; I absolutely would not</em> write any substantial Rust project without it at this point.</p>
Inspirations</h1>
I wouldn’t have gotten even this far on scum</code>2</a></sup> without the help of</p>

mal</code></a>,</li>
“Write you a Scheme,” the 2.0 update</a> and the OG version</a>,</li>
SICP, both the book</a> and the video lectures</a>,</li>
Peter Norvig’s beautifully</a> clean Python</a> implementations.</li>
</ul>
Postscript</h1>
👋 to the former coworker who enjoined me to blog more!
I miss working with you, friend.</p>



It’s the exact opposite</em> of raganwald</code>’s</a> README section, inspiration here</a>. ↩</a></p>
</li>

I didn’t consume all of this recently in my quest to put scum</code> together, but I would almost certainly be remiss in not mentioning these; in fact, I’m probably leaving lots out! ↩</a></p>
</li>
</ol>
</section>


Algebraic Structure ⇒ Computational Benefits, Now With 100% More Oxidization
2022-09-13T00:00:00+00:00
Code or it didn’t happen</h1>
I’ve been noodling with a Rust implementation of the ideas from this
talk</a>, and decided I should probably
share it—or at least put it up online in case my laptop crashes. The repo is
available here</a>.</p>


Time Traveling Key-Value Store, This Time in Rust
2022-04-21T00:00:00+00:00
Previously…</h1>
We’ve talked</a>
previously</a> about implementing a time-traveling
key-value store.
Having worked more with Rust</a> in the interim, I
tried my hand at a Rust implementation.
Rust’s type system, standard library, and attention to pedantic details make
this my favorite version yet.</p>
Implementation</h1>
The standard library provides everything we need:</p>
use</span> std</span>::</span>collections</span>::</span>BTreeMap</span>;</span></span>
use</span> std</span>::</span>time</span>::</span>Instant</span>;</span></span>
</span>
pub struct</span> Ttkv</span><</span>K</span>,</span> V</span>> {</span></span>
    started</span>:</span> Instant</span>,</span></span>
    mapping</span>:</span> BTreeMap</span><</span>u128</span>, (</span>K</span>,</span> V</span>)>,</span></span>
}</span></span></code></pre>
Recording the std::time::Instant</code> when we created our store allows us to
monotonically record insertion timestamps—on most hardware, at least—and we
can still check if this assumption is violated.</p>
We build a new store via implementing Default</code>:</p>
impl</span><</span>K</span>,</span> V</span>></span> Default</span> for</span> Ttkv</span><</span>K</span>,</span> V</span>> {</span></span>
    fn</span> default</span>()</span> -></span> Self</span> {</span></span>
        Self</span> {</span></span>
            started</span>:</span> Instant</span>::</span>now</span>(),</span></span>
            mapping</span>:</span> BTreeMap</span>::</span>default</span>(),</span></span>
        }</span></span>
    }</span></span>
}</span></span></code></pre>
Digging into the implementation (the following snippets are all wrapped in an
impl</code> block), it’s straightforward to check for emptiness:</p>
pub fn</span> is_empty</span>(</span>&</span>self</span>)</span> -></span> bool</span> {</span></span>
    self</span>.</span>mapping</span>.</span>is_empty</span>()</span></span>
}</span></span></code></pre>
Adding a pair to our store gives us an opportunity to assert that time is
monotonic:</p>
pub fn</span> put</span>(</span>&mut</span> self</span>, key</span>:</span> K</span>, value</span>:</span> V</span>, timestamp</span>:</span> Option</span><</span>u128</span>>) {</span></span>
    let</span> t</span> =</span> timestamp</span>.</span>unwrap_or_else</span>(</span>||</span> {</span></span>
        Instant</span>::</span>now</span>()</span></span>
            .</span>checked_duration_since</span>(</span>self</span>.</span>started)</span></span>
            .</span>unwrap_or_else</span>(</span>||</span> panic!</span>(</span>"non-monotonic insertion"</span>))</span></span>
            .</span>as_nanos</span>()</span></span>
    });</span></span>
    self</span>.</span>mapping</span>.</span>insert</span>(t, (key, value));</span></span>
}</span></span></code></pre>
Retrieval from the store is similar to previous implementations:</p>
pub fn</span> get</span>(</span>&</span>self</span>, key</span>: &</span>K</span>, timestamp</span>:</span> Option</span><</span>u128</span>>)</span> -></span> Option</span><</span>&</span>V</span>></span></span>
where</span></span>
    K</span>:</span> PartialEq</span>,</span></span>
{</span></span>
    self</span>.</span>mapping</span></span>
        .</span>range</span>(</span>0</span>..</span>timestamp</span>.</span>unwrap_or</span>(</span>u128</span>::</span>MAX</span>))</span></span>
        .</span>filter</span>(</span>|</span>(_, (k, _))</span>|</span> k</span> ==</span> key)</span></span>
        .</span>last</span>()</span></span>
        .</span>map</span>(</span>|</span>(_, (_, v))</span>|</span> v)</span></span>
}</span></span></code></pre>
Finally, collecting the insertion times in order:</p>
pub fn</span> times</span>(</span>&</span>self</span>)</span> -></span> Vec</span><</span>u128</span>> {</span></span>
    self</span>.</span>mapping</span>.</span>keys</span>()</span>.</span>cloned</span>()</span>.</span>collect</span>()</span></span>
}</span></span></code></pre>Testing</h1>
Rust’s (and cargo</code>’s) approach to testing is a pleasure to use.
We can do regular unit-style tests:</p>
#[test]</span></span>
fn</span> initially_empty</span>() {</span></span>
    let</span> t</span> =</span> Ttkv</span>::</span><</span>String</span>,</span> String</span>></span>::</span>default</span>();</span></span>
    assert!</span>(t</span>.</span>is_empty</span>());</span></span>
    assert!</span>(t</span>.</span>times</span>()</span>.</span>is_empty</span>());</span></span>
}</span></span></code></pre>
And with the help of
proptest</code></a> we can do
property-based testing:</p>
#[test]</span></span>
fn</span> two_gets_different_keys</span>(a</span>:</span> String</span>, b</span>:</span> String</span>, x</span>:</span> String</span>, y</span>:</span> String</span>) {</span></span>
    prop_assume!</span>(a</span> !=</span> b);</span></span>
    let mut</span> t</span> =</span> Ttkv</span>::</span>default</span>();</span></span>
    t</span>.</span>put</span>(a</span>.</span>clone</span>(), x</span>.</span>clone</span>(),</span> None</span>);</span></span>
    t</span>.</span>put</span>(b</span>.</span>clone</span>(), y</span>.</span>clone</span>(),</span> None</span>);</span></span>
    prop_assert_eq!</span>(t</span>.</span>times</span>()</span>.</span>len</span>(),</span> 2</span>);</span></span>
    prop_assert_eq!</span>(t</span>.</span>get</span>(</span>&</span>a,</span> None</span>),</span> Some</span>(</span>&</span>x));</span></span>
    prop_assert_eq!</span>(t</span>.</span>get</span>(</span>&</span>b,</span> None</span>),</span> Some</span>(</span>&</span>y));</span></span>
}</span></span></code></pre>
As a side note, this test originally failed for me in an early draft where the
timestamping mechanism wasn’t monotonic.</p>
Fin.</h1>
This was a fun exercise, and again is probably my favorite implementation.
See the repo</a> for the full
code!</p>


Synthetic weather radar using hybrid quantum-classical machine learning
2021-12-01T00:00:00+00:00
New paper</h1>
I’m excited to announce that our paper on using quantum ML to predict
weather radar products is up on the arXiv</a>!
What’s more, per the press
release</a>,
we’ll be presenting the paper at the Artificial Intelligence for Humanitarian
Assistance and Disaster Response Workshop</a> at NeurIPS
2021</a>.</p>


CvRDT Exposition in Rust
2020-09-01T00:00:00+00:00
Programming with types for understanding</h1>
One of the best uses of the type system I’ve seen is the Build systems a la carte</a> paper.
In it, the authors use Haskell’s types to outline and explore the problem domain in really novel ways.
I wanted to understand CRDTs</a> more, so I made a thing.
It’s nowhere near the level of Build systems a la carte</em>, but I found it useful.</p>
Understanding CvRDTs</h1>
CRDTs are important to modern distributed programming; as the above Wikipedia article explains, they are</p>

data structure[s] which can be replicated across multiple computers in a network, where the replicas can be updated independently and concurrently without coordination between the replicas, and where it is always mathematically possible to resolve inconsistencies that might come up.</p>
</blockquote>
In reading about CRDTs, I came across the phenomenal paper A comprehensive study of Convergent and Commutative Replicated Data Types</a>.
With this paper, Wikipedia, and some other references as my guide, I made a Rust crate</a> to understand state-based CRDTs (a.k.a. convergent replicated data types or CvRDTs).
I really like how this crate uses two traits with associated types to describe CvRDTs, fitting them into a common framework.</p>
The first trait is for CvRDTs that can only grow, i.e. only add items.
Here’s an slightly modified version of it; the code (on GitHub</a>, with documentation on docs.rs</a>) has more:</p>
pub trait</span> Grow</span>:</span> Clone</span> {</span></span>
    type</span> Payload</span>:</span> Eq</span>;</span>                                    // Internal state</span></span>
    type</span> Update</span>;</span>                                         // Message to update internal state</span></span>
    type</span> Query</span>;</span>                                          // Query message</span></span>
    type</span> Value</span>;</span>                                          // Query response</span></span>
</span>
    fn</span> new</span>(payload</span>:</span> Self</span>::</span>Payload</span>)</span> -></span> Self</span>;</span>              // Create a new version of our CvRDT from Payload</span></span>
    fn</span> payload</span>(</span>&</span>self</span>)</span> -></span> Self</span>::</span>Payload</span>;</span>                  // Retrieve Payload</span></span>
    fn</span> add</span>(</span>&mut</span> self</span>, update</span>:</span> Self</span>::</span>Update</span>);</span>             // Add item, mutating in-place</span></span>
    fn</span> le</span>(</span>&</span>self</span>, other</span>: &</span>Self</span>)</span> -></span> bool</span>;</span>                  // Is this ≤ other in semilattice's partial order?</span></span>
    fn</span> merge</span>(</span>&</span>self</span>, other</span>: &</span>Self</span>)</span> -></span> Self</span>;</span>               // Merge this and other into new CvRDT</span></span>
    fn</span> query</span>(</span>&</span>self</span>, query</span>: &</span>Self</span>::</span>Query</span>)</span> -></span> Self</span>::</span>Value</span>;</span> // Query to get some Value</span></span>
}</span></span></code></pre>
Our rigorous typing does make for one difference from the aforementioned paper</a> and Wikipedia.
When implementing GCounter</code>s</a> and PNCounter</code>s</a>, we no longer have an unspecified myId()</code> function giving the current node’s index.
In order for the Payload</code> type to fully specify their internal state, these classes require that index be part of the payload.</p>
Other CvRDTs can also shrink, i.e. remove items.
Here’s a slightly modified version of cvrdt-exposition</code>’s trait:</p>
pub trait</span> Shrink</span>:</span> Grow</span> {</span></span>
    fn</span> del</span>(</span>&mut</span> self</span>, update</span>:</span> Self</span>::</span>Update</span>);</span>             // Delete item, mutating in-place</span></span>
}</span></span></code></pre>Testing and Verification</h1>
We have the Eq</code> bound on the Payload</code> type for verification and testing.
In order to verify that my CvRDT implementations behave as required, i.e. that their merge functions are commutative, associative, and idempotent, I turned to the proptest</code></a> crate for property-based testing, via assert_eq!</code> calls.
I turned to Rust’s macro system to generate some of these tests for me; see properties.rs</code></a> in the GitHub source and the test</code> sub-modules of the implementation modules</a> for more.</p>


Tiny REST API Demo in Python
2020-08-17T00:00:00+00:00
Introduction</h1>
The other day, someone asked me about the difference between “an API and a regular webpage.”
After understanding more about the context of their question, I tried to come up with a decent explanation about the differences between a server sending HTML pages and one handling REST API requests.
As a thousand words leave not the same deep impression as does a single deed</a>, I thought I’d put together a tiny demonstration that serves HTML to a web browser and JSON over a REST endpoint.
My demo uses the Flask</a> microframework to handle the details; you can find a copy here</a>, though we’ll go through some of it below.</p>
How smol?</h1>
Before we get into the specifics, here’s how tiny the demo is: the whole thing, including a flake.nix</code> file to set the stage, is under 115 lines.
Via tokei</code></a>:</p>
===============================================================================</span></span>
 Language            Files        Lines         Code     Comments       Blanks</span></span>
===============================================================================</span></span>
 CSS                     1           10           10            0            0</span></span>
 HTML                    3           41           41            0            0</span></span>
 Markdown                1           10            0            6            4</span></span>
 Nix                     1           25           23            0            2</span></span>
 Python                  2           54           40            2           12</span></span>
===============================================================================</span></span>
 Total                   8          140          114            8           18</span></span>
===============================================================================</span></span></code></pre>
This whole thing cuts so many corners it might as well be a sphere.
It’s certainly</em> not a production-ready example by any</em> stretch of the imagination; it’s only enough for demonstration purposes.</p>
Preliminaries</h1>
First, we need some data.
Pretend we’re creating the School Reporter 5000™</em>, wherein we have student information that consists of a collection of (timestamp, person, activity)</code> tuples.
In data.py</code>, we generate some fake data.
Our application code will interface with this data via NAMES</code> and SELECT</code>; pretend this stands in for a proper database.</p>
from</span> datetime</span> import</span> datetime, timedelta</span></span>
from</span> itertools</span> import</span> accumulate</span></span>
from</span> operator</span> import</span> add</span></span>
import</span> random</span></span>
</span>
random.seed(</span>1729</span>)</span></span>
</span>
NAMES</span> =</span> [</span>"Alice"</span>,</span> "Bob"</span>,</span> "Carol"</span>,</span> "Dave"</span>]</span></span>
_ACTIVITIES</span> =</span> [</span>"Reading"</span>,</span> "Writing"</span>,</span> "Arithmetic"</span>]</span></span>
_DATA</span> =</span> [</span></span>
    {</span></span>
        "timestamp"</span>: ts.time().isoformat(</span>timespec</span>=</span>"minutes"</span>),</span></span>
        "person"</span>: random.choice(</span>NAMES</span>),</span></span>
        "activity"</span>: random.choice(</span>_ACTIVITIES</span>),</span></span>
    }</span></span>
    for</span> ts</span> in</span> accumulate(</span></span>
        (timedelta(</span>minutes</span>=</span>random.randrange(</span>17</span>))</span> for</span> _</span> in</span> range</span>(</span>42</span>)),</span></span>
        func</span>=</span>add,</span></span>
        initial</span>=</span>datetime.today().replace(</span>hour</span>=</span>9</span>),</span></span>
    )</span></span>
]</span></span>
SELECT</span> =</span> {name: [x</span> for</span> x</span> in</span> _DATA</span> if</span> x[</span>"person"</span>]</span> ==</span> name]</span> for</span> name</span> in</span> NAMES</span>}</span></span></code></pre>App</h1>
The School Reporter 5000™</em> has three routes, all described in app.py</code>.
We serve HTML for the homepage at /index.html</code> and individual person pages at /person/<name></code> for each of the names in NAMES</code>.
We also serve JSON in response to GET requests via the /api/<name></code> route.
Thanks to Flask</a>, we even have rudimentary error handling.</p>
from</span> flask</span> import</span> Flask, jsonify, render_template</span></span>
from</span> data</span> import</span> NAMES</span>,</span> SELECT</span></span>
</span>
app</span> =</span> Flask(</span>__name__</span>)</span></span>
</span>
@app.route</span>(</span>"/"</span>)</span></span>
def</span> home</span>():</span></span>
    return</span> render_template(</span>"index.html"</span>,</span> names</span>=</span>NAMES</span>)</span></span>
</span>
@app.route</span>(</span>"/person/<string:name>"</span>)</span></span>
def</span> person</span>(name):</span></span>
    try</span>:</span></span>
        return</span> render_template(</span>"person.html"</span>,</span> name</span>=</span>name,</span> data</span>=</span>SELECT</span>[name],</span> error</span>=</span>False</span>)</span></span>
    except</span> KeyError</span>:</span></span>
        return</span> render_template(</span>"person.html"</span>,</span> name</span>=</span>name,</span> error</span>=</span>True</span>),</span> 400</span></span>
</span>
@app.route</span>(</span>"/api/<string:name>"</span>)</span></span>
def</span> api</span>(name):</span></span>
    try</span>:</span></span>
        return</span> jsonify(</span>SELECT</span>[name])</span></span>
    except</span> KeyError</span>:</span></span>
        return</span> {</span>"error"</span>:</span> f</span>"Unknown person </span>{</span>name</span>}</span>"</span>},</span> 400</span></span></code></pre>
With the server running, we can visit the home page:</p>
</p>
Or report pages for any specific student:</p>
</p>
However, we can also get JSON data for individual students by pinging the REST endpoint; using, e.g. HTTPie</a> to query the endpoint via http --body :5000/api/Dave</code>, we get</p>
</p>
Everything Else</h1>
To round out the example, we also have some minimal CSS</a> and two other HTML templates</a>.
Here’s the flake.nix</code> to specify the W O R L D:</p>
{</span></span>
  description</span> =</span> "Tiny API Demo"</span>;</span></span>
</span>
  inputs</span> =</span> {</span></span>
    flake-utils</span>.</span>url</span> =</span> "github:numtide/flake-utils"</span>;</span></span>
    nixpkgs</span>.</span>url</span> =</span> "github:NixOS/nixpkgs/nixos-22.11"</span>;</span></span>
  };</span></span>
</span>
  outputs</span> =</span> {</span></span>
    self</span>,</span></span>
    flake-utils</span>,</span></span>
    nixpkgs</span>,</span></span>
  }:</span></span>
    flake-utils</span>.</span>lib</span>.</span>eachDefaultSystem</span> (system:</span> let</span></span>
      pkgs</span> =</span> import</span> nixpkgs</span> {</span>inherit</span> system</span>;};</span></span>
      py</span> =</span> pkgs</span>.</span>python310</span>.</span>withPackages</span> (p: [</span>p</span>.</span>flask</span>]);</span></span>
    in</span> {</span></span>
      packages</span>.</span>default</span> =</span> pkgs</span>.</span>writeShellApplication</span> {</span></span>
        name</span> =</span> "tiny_api_demo"</span>;</span></span>
        text</span> =</span> ''</span></span>
          FLASK_APP=app.py ${</span>py</span>}/bin/python -m flask run</span></span>
        ''</span>;</span></span>
      };</span></span>
    });</span></span>
}</span></span></code></pre>
From the base directory, one can run nix run</code> to set up the environment and kick off the server.</p>


PostgreSQL Makes Delicious Dog Food
2020-06-23T00:00:00+00:00
Introduction</h1>
It’s probably not news to you, but folks behind the PostgreSQL
Database</a> have made an incredible</em> piece of
software. It’s fast, it can handle loads of data, and it has interesting
builtin functions and index types for almost anything you might need. For this
post, though, I’d like to highlight how it facilitates DB maintenance tasks by
eating its own dog
food</a>—that is, how
answering questions about</em> the database is no different from running regular
SQL queries.</p>
Problem statement</h1>
At work</a>, we’ve got a fair amount of data. I’m
charged with the care and feeding of a couple of these databases, one of which
recently got a huge</em> upgrade of incoming data. Previously it contained
our data vendor’s restricted version of US-and-Canada-only data; we switched to
the WHOLE_DANG_WORLD</code> option.</p>
When we (I) created our DB schema, we relied heavily upon materialized views;
the thought was we’d keep the incoming raw data in tables, perform some
DB-level ETL work, then put the main data we wish to use into tidy,
consolidated materialized views. This design made sense given what we thought
our paradigm was going to be: we were expecting</p>

raw data that needed extensive work before being suitable for processing, and</li>
we would receive small incremental updates of that data frequently from our vendor, so we could UPDATE</code> the raw tables and REFRESH</code> the materialized views to stay up to date.</li>
</ul>
However, this is decidedly not the paradigm in which we find ourselves. The
raw data is almost ready for our ML algorithms to grab, modulo some
normalizations, and instead of frequent small updates we receive quarterly
dumps of the entire</em> database. This led to a lot of waste, but there was one
particular pain point: because we used materialized views, when our intrepid
Ops team copied the DB over to other environments (e.g. stage & prod), they had
to rebuild every index</em> on the materialized views—unlike a table index, an
index on a materialized view doesn’t copy over.</p>
I had a strong suspicion that we didn’t need every index we were copying over;
over half are obsolete artifacts of query patterns we no longer used. I thought
of combing through our Scala monolith to find every time we accessed these
materialized views via Slick</a>, but calling that
error-prone would be an understatement. Part of the appeal of Slick is that it
turns normal-looking Scala code into SQL queries for you, but that means you
need to translate it—either in code by, say, debug logging the query statements
or in your head—to fully understand what it’s doing with your DB.</p>
It’s tables all the way down</h1>
Luckily, the PostgreSQL devs have made answering these types of questions
manageable. One answers questions about PostgreSQL by, well, querying PostgreSQL.
There are a bevy of system
catalogs</a> and
collected statistics
views</a>
chock-full of helpful information.</p>
Trimming down an example query from the wiki page on index
maintenance</a>, I ran the
following to get the names, table names, and human-readable sizes of the
indexes that were</p>

public (d.schemaname = 'public'</code>), meaning we made them,</li>
not unique (not a.indisunique</code>), so they’re not primary keys1</a></sup>, and</li>
never scanned (d.idx_scan = 0</code>), so they’re unused:</li>
</ul>
select</span></span>
  c</span>.</span>relname</span> as</span> index_name,</span></span>
  b</span>.</span>relname</span> as</span> table_name,</span></span>
  pg_size_pretty(pg_relation_size(</span></span>
    quote_ident(</span>d</span>.</span>schemaname</span>) </span>||</span> '.'</span> ||</span> quote_ident(</span>d</span>.</span>indexrelname</span>)</span></span>
  )) </span>as</span> index_size</span></span>
from</span> pg_index a</span></span>
join</span> pg_class b </span>on</span> b</span>.</span>oid</span> =</span> a</span>.</span>indrelid</span></span>
join</span> pg_class c </span>on</span> c</span>.</span>oid</span> =</span> a</span>.</span>indexrelid</span></span>
join</span> pg_stat_all_indexes d </span>on</span> a</span>.</span>indexrelid</span> =</span> d</span>.</span>indexrelid</span></span>
where</span> d</span>.</span>schemaname</span> =</span> 'public'</span> and not</span> a</span>.</span>indisunique</span> and</span> d</span>.</span>idx_scan</span> =</span> 0</span></span>
order by</span> table_name, index_name;</span></span></code></pre>
When run, this query immediately returns something like2</a></sup></p>
|   index_name   |   table_name   |   index_size   |</span></span>
|----------------|----------------|----------------|</span></span>
|   unused_idx   |   a_table      |   1.81 GB      |</span></span>
|   also_bad     |   another_tbl  |   77 MB        |</span></span></code></pre>
With this helpful query in hand, I knew exactly which indexes we could safely
delete and how much memory we’d save the database by doing so.</p>



Or the approximate equivalent for materialized views, which can’t have primary indexes, but can have unique indexes. ↩</a></p>
</li>

Names changed to protect the guilty. ↩</a></p>
</li>
</ol>
</section>


Silly Shenanigans
2019-08-14T00:00:00+00:00
Introduction</h1>
In an effort to both post more than once a year and clean out some silly things
from my ~/tmp</code> directory, here’s a quick post about two programming
shenanigans.
Perhaps unsurprisingly due to its history of jocularity, both originate with
Perl.</p>
Bleach</h1>
Damian Conway</a> is a truly prolific
member of the Perl community.
One of Conway’s less serious contributions is the
Acme::Bleach</code></a> module, famous for
being the first Acme</code> module and the first source filter joke
module.</a>
Acme::Bleach</code> takes programs that look like this:</p>
use</span> Acme::Bleach;</span></span>
print</span> "Hello world"</span>;</span></span></code></pre>
and turns them into</p>
use</span> Acme::Bleach;</span></span>
 	 	 	 	 	 	 	 			   	   </span></span>
 		 		 	 </span></span>
 	 		  		</span></span>
	 		 	 	 </span></span>
 		      </span></span>
	  	   		</span></span>
       	 </span></span>
 			 			 </span></span>
 			 	   </span></span>
   			   </span></span>
		 		  	 </span></span>
	        </span></span>
			  	  	</span></span>
		 	  	 	</span></span>
	  			 		</span></span>
   	 			 </span></span>
     	   </span></span>
	   	    </span></span>
 	  	 	 	</span></span>
  		   		</span></span>
 		   		 </span></span>
		 				 	</span></span>
	      	 </span></span>
 			 			 </span></span>
				 		  </span></span>
	  			   </span></span>
		 		   	</span></span>
  		  	  </span></span>
 	  		 		</span></span>
	   	 	  </span></span>
  </span></span></code></pre>
No, that’s not an error.
Through the magic of Perl’s source
filters</a>, the above program whose
source consists of nothing but one import statement and a bunch of whitespace
still prints “Hello world”!</em></p>
After digging through the source, I decided to try my hand at recreating it in
Python:</p>
#!/usr/bin/env python3</span></span>
"""For really clean programs"""</span></span>
</span>
from</span> functools</span> import</span> reduce</span></span>
from</span> pathlib</span> import</span> Path</span></span>
from</span> subprocess</span> import</span> run</span></span>
from</span> sys</span> import</span> argv</span></span>
</span>
ZERO</span> =</span> ord</span>(</span>" "</span>)</span></span>
ONE</span> =</span> ord</span>(</span>"</span>\t</span>"</span>)</span></span>
</span>
def</span> zero_one</span>(x:</span> int</span>) -></span> bytes</span>:</span></span>
    """Convert a byte to a series of `ZERO` and `ONE`"""</span></span>
    return</span> bytes</span>([</span>ZERO</span>,</span> ONE</span>][(x</span> >></span> i)</span> &</span> 1</span>]</span> for</span> i</span> in</span> range</span>(</span>7</span>,</span> -</span>1</span>,</span> -</span>1</span>))</span></span>
</span>
def</span> un_zero_one</span>(xs:</span> bytes</span>) -></span> int</span>:</span></span>
    """Undo `zero_one`"""</span></span>
    return</span> reduce</span>(</span>lambda</span> i, x: (i</span> <<</span> 1</span>)</span> |</span> int</span>(x</span> ==</span> ONE</span>), xs,</span> 0</span>)</span></span>
</span>
def</span> encode</span>(xs:</span> bytes</span>) -></span> bytes</span>:</span></span>
    """Use `zero_one` to encode"""</span></span>
    return b</span>""</span>.join(zero_one(x)</span> for</span> x</span> in</span> xs)</span></span>
</span>
def</span> decode</span>(xs:</span> bytes</span>) -></span> bytes</span>:</span></span>
    """Use `un_zero_one` to decode"""</span></span>
    return</span> bytes</span>(un_zero_one(xs[i : i</span> +</span> 8</span>])</span> for</span> i</span> in</span> range</span>(</span>0</span>,</span> len</span>(xs),</span> 8</span>))</span></span>
</span>
if</span> __name__</span> ==</span> "__main__"</span>:</span></span>
    if</span> len</span>(argv)</span> !=</span> 2</span>:</span></span>
        exit</span>(</span>"I need a file to bleach"</span>)</span></span>
    else</span>:</span></span>
        XS</span> =</span> Path(argv[</span>1</span>]).read_bytes()</span></span>
        if</span> any</span>(x</span> not in</span> {</span>ZERO</span>,</span> ONE</span>}</span> for</span> x</span> in</span> XS</span>):</span></span>
            print</span>(encode(</span>XS</span>).decode(</span>"ascii"</span>),</span> end</span>=</span>""</span>,</span> flush</span>=</span>True</span>)</span></span>
        else</span>:</span></span>
            run([</span>"python3"</span>,</span> "-"</span>],</span> input</span>=</span>decode(</span>XS</span>))</span></span></code></pre>
Since Python doesn’t have Perl’s source filters, we can’t do everything that
Acme::Bleach</code> does, but bleach.py</code> uses the same main trick: source code
consists of bytes, as far as computers care, so we can encode and decode them
however we want—including using spaces for zeros and tabs for ones.
If we have a file called simple.py</code> consisting of print("hello world")</code>,
we can say ./bleach.py simple.py > simple_bleached.py</code>.
Then running ./bleach.py simple_bleached.py</code> will result in the interpreter
printing "hello world"</code>.</p>
D&D</h1>
I recently came across a fun
post</a> that describes
implementing basic dice notation from Dungeons & Dragons in Perl 6.
It’s a fun read, so you should definitely check it out, but its basic
implementation looks like this:</p>
sub</span> infix</span>:<d>(Int $n, Int $max) { (1..$max).pick xx $n }</span></span></code></pre>
So one can enter say (3 d 20);</code> and see something like (17 1 12)</code> printed.
Haskell has nice support for infix-ing any function by wrapping it in
backticks, so I came up with the following dnd.hs</code>:</p>
#!/usr/bin/env stack</span></span>
-- stack --install-ghc runghc --package random --resolver lts-14.0</span></span>
module</span> Main</span> where</span></span>
</span>
import</span> Data.Foldable</span> (</span>traverse_</span>)</span></span>
import</span> System.Random</span> (</span>getStdGen</span>,</span> randomRs</span>)</span></span>
</span>
d</span> :: Word -> Word -> IO</span> [</span>Word</span>]</span></span>
d n size </span>=</span> take (fromIntegral n)</span> .</span> randomRs (</span>1</span>, size)</span> <$></span> getStdGen</span></span>
</span>
main</span> :: IO</span> ()</span></span>
main </span>=</span> traverse_ (print </span>=<<</span>) [</span>3</span> `d`</span> 8</span>,</span> 2</span> `d`</span> 20</span>,</span> 1</span> `d`</span> 6</span>]</span></span></code></pre>
Running it via ./dnd.hs</code>1</a></sup>, I got</p>
[</span>6</span>,</span>6</span>,</span>3</span>]</span></span>
[</span>10</span>,</span>10</span>]</span></span>
[</span>2</span>]</span></span></code></pre>
Though of course different runs will probably output different results.</p>



Assuming of course that you’ve got stack</code></a> installed. ↩</a></p>
</li>
</ol>
</section>


Time Traveling Key-Value Store, Again
2019-08-05T00:00:00+00:00
Introduction</h1>
In my last post</a> I talked
about a time traveling key-value store implementation in some functional Scala.
After thinking about the problem more—probably a little too
much</a>—I decided to try my hand at another
implementation, this time in Python again.
Rather than going the purely functional route, though, I wanted to focus on how
we represent the data internally.
Since we essentially have tabular data with three columns (timestamp, key, and
value), it made some sense to try and work with a table via the sqlite3</code>
module in the Python standard
library</a>.</p>
Implementation</h1>
First, some imports:</p>
from</span> pickle</span> import</span> dumps, loads</span></span>
from</span> sqlite3</span> import</span> Row, connect</span></span>
from</span> time</span> import</span> perf_counter</span></span>
from</span> typing</span> import</span> Any, List, Optional</span></span></code></pre>
We use the pickle</code> module to marshal (almost) arbitrary keys and values to
binary blobs, time</code> to give us a global clock as in my in-the-room
implementation from the interview, and typing</code> to label our types.</p>
Since I wanted to wrap all this up with a stable release of
nix</code></a>, I needed to use Python 3.6; this meant that
time.perf_counter_ns()</code> wasn’t available, so we make our own out of
time.perf_counter()</code>:</p>
def</span> perf_counter_ns</span>() -></span> int</span>:</span></span>
    return</span> int</span>(perf_counter()</span> *</span> 1e9</span>)</span></span></code></pre>
If one feels really strongly about Python 3.7, one could use the unstable
nixpkgs</code> channel.</p>
Next we create a TTKV</code> class:</p>
class</span> TTKV</span>:</span></span>
</span>
    def</span> __init__</span>(self) -></span> None</span>:</span></span>
        self</span>.conn</span> =</span> connect(</span>":memory:"</span>)</span></span>
        self</span>.conn.row_factory</span> =</span> Row</span></span>
        self</span>.cursor</span> =</span> self</span>.conn.cursor()</span></span>
        self</span>.cursor.execute(</span></span>
            "create table ttkv (timestamp integer, key blob, value blob)"</span></span>
        )</span></span>
        self</span>.cursor.execute(</span>"create index timestamp_index on ttkv(timestamp)"</span>)</span></span>
        self</span>.cursor.execute(</span>"create index key_index on ttkv(key)"</span>)</span></span>
        self</span>.conn.commit()</span></span></code></pre>
Using this in-memory sqlite</code> db, put()</code> is an INSERT</code> statement:</p>
    def</span> put</span>(self, key: Any, value: Any) -></span> None</span>:</span></span>
        self</span>.cursor.execute(</span></span>
            "insert into ttkv values (?, ?, ?)"</span>,</span></span>
            (perf_counter_ns(), dumps(key), dumps(value)),</span></span>
        )</span></span></code></pre>
While a SELECT</code> statement takes care of get()</code>:</p>
    def</span> get</span>(self, key: Any, timestamp: Optional[</span>int</span>]</span> =</span> None</span>) -> Any:</span></span>
        select</span> =</span> "select * from ttkv where key = ?"</span></span>
        order</span> =</span> "order by timestamp desc"</span></span>
        if</span> timestamp</span> is</span> None</span>:</span></span>
            self</span>.cursor.execute(</span>f</span>"</span>{</span>select</span>} {</span>order</span>}</span>"</span>, (dumps(key),))</span></span>
        elif</span> isinstance</span>(timestamp,</span> int</span>):</span></span>
            self</span>.cursor.execute(</span></span>
                f</span>"</span>{</span>select</span>}</span> and timestamp <= ? </span>{</span>order</span>}</span>"</span>, (dumps(key), timestamp)</span></span>
            )</span></span>
        else</span>:</span></span>
            raise</span> ValueError</span>(timestamp)</span></span>
        result</span> =</span> self</span>.cursor.fetchone()</span></span>
        if</span> result</span> is</span> None</span>:</span></span>
            raise</span> KeyError</span>(key)</span></span>
        return</span> loads(result[</span>"value"</span>])</span></span></code></pre>
To be more Pythonic, get()</code> throws errors if given an unknown key or a
non-integral timestamp rather than going the Scala version’s Option</code> route.</p>
Finally, the times()</code> helper returns relevant timestamps like in the Scala
version:</p>
    def</span> times</span>(self, key: Any</span> =</span> None</span>) -> List[</span>int</span>]:</span></span>
        select</span> =</span> "select timestamp from ttkv"</span></span>
        order</span> =</span> "order by timestamp desc"</span></span>
        if</span> key</span> is</span> None</span>:</span></span>
            self</span>.cursor.execute(</span>f</span>"</span>{</span>select</span>} {</span>order</span>}</span>"</span>)</span></span>
        else</span>:</span></span>
            self</span>.cursor.execute(</span>f</span>"</span>{</span>select</span>}</span> where key = ? </span>{</span>order</span>}</span>"</span>, (dumps(key),))</span></span>
        return</span> [row[</span>"timestamp"</span>]</span> for</span> row</span> in</span> self</span>.cursor.fetchall()]</span></span></code></pre>Testing</h1>
My
ttkv_spec.py</code></a>
looks similar to the previous
TTKVSpec.scala</code></a>,
using the wonderful hypothesis</code>
library</a> in place of scalacheck</code>
for property testing.
I also lean on pytest</code></a> for running
the tests and for handling (deliberate) exceptions.
I’ll copy just three tests here; interested readers should check the source on
GitHub for the rest.</p>
@given</span>(</span>STR</span>)</span></span>
def</span> test_initially_empty</span>(a:</span> str</span>) -></span> None</span>:</span></span>
    with</span> raises(</span>KeyError</span>):</span></span>
        TTKV().get(a)</span></span>
</span>
@given</span>(distinct_keys(),</span> INT</span>,</span> INT</span>)</span></span>
def</span> test_two_gets_different_keys</span>(ab: Tuple[</span>str</span>,</span> str</span>], x:</span> int</span>, y:</span> int</span>) -></span> None</span>:</span></span>
    a, b</span> =</span> ab</span></span>
    ttkv</span> =</span> TTKV()</span></span>
    ttkv.put(a, x)</span></span>
    ttkv.put(b, y)</span></span>
    assert</span> ttkv.get(a)</span> ==</span> x</span></span>
    assert</span> ttkv.get(b)</span> ==</span> y</span></span>
</span>
@given</span>(</span>STR</span>,</span> INT</span>, integers(</span>min_value</span>=</span>0</span>,</span> max_value</span>=</span>SQLITE_MAX_INT</span>))</span></span>
def</span> test_get_before_time</span>(a:</span> str</span>, x:</span> int</span>, t:</span> int</span>) -></span> None</span>:</span></span>
    ttkv</span> =</span> TTKV()</span></span>
    ttkv.put(a, x)</span></span>
    t0</span> =</span> ttkv.times(a)[</span>0</span>]</span></span>
    with</span> raises(</span>KeyError</span>):</span></span>
        TTKV().get(a, t0</span> -</span> t)</span></span></code></pre>Thoughts</h1>
I’m of two minds here.
Like the in-the-interview version I mentioned in my last post, this code relies
heavily on side-effects, using  its global nanosecond clock for timestamp
generation.
I liked how the functional Scala version explicitly isolated the effectful
put()</code> computation.
However, I like how easily we map the TTKV</code> API onto the sqlite</code> table; the
ORDER BY</code> clauses obviate the need to sort things, e.g.</p>
It also seems that if the spec for TTKV</code> changes at all (as software specs are
wont to do), we could easily modify this approach.
It also suggests using multiple data backends, wherein we might define an
abstract API and connect it to different specific data structures depending on
our needs.</p>
As a happy accident of using the pickle</code> module, this TTKV</code> implementation
can use more types of things as keys than even the base dict</code> class (e.g.
lists).
There are restrictions on what can be
pickled</a>,
of course, and this implementation doesn’t handle any PicklingError</code>s that
might occur if a use tries to save particular exotic keys or values.</p>
I’m a huge</em> fan of
hypothesis</code></a>,
pytest</code></a>, and
typing</code></a> along with
mypy</code></a>; I’d go so far as to say that I’d be reticent to
develop serious software in Python without them.
One (perhaps) interesting thing that happened in testing: though the sqlite3</code>
module’s documentation claims that one can make arbitrary Python integers into
native sqlite</code> integer values, hypothesis</code> quickly exposed a flaw in that
claim, running into OverflowError</code>s when trying to save integral values too
large to fit into 64 bits.</p>
Appendix</h1>
Here’s the flake.nix</code> file to specify the W O R L D:</p>
{</span></span>
  description</span> =</span> "Python TTKV"</span>;</span></span>
</span>
  inputs</span> =</span> {</span></span>
    flake-utils</span>.</span>url</span> =</span> "github:numtide/flake-utils"</span>;</span></span>
    nixpkgs</span>.</span>url</span> =</span> "github:NixOS/nixpkgs/nixos-23.11"</span>;</span></span>
  };</span></span>
</span>
  outputs</span> =</span> {</span></span>
    self</span>,</span></span>
    flake-utils</span>,</span></span>
    nixpkgs</span>,</span></span>
  }:</span></span>
    flake-utils</span>.</span>lib</span>.</span>eachDefaultSystem</span> (system:</span> let</span></span>
      pkgs</span> =</span> import</span> nixpkgs</span> {</span>inherit</span> system</span>;};</span></span>
      python</span> =</span> pkgs</span>.</span>python311</span>.</span>withPackages</span> (p: [</span>p</span>.</span>hypothesis p</span>.</span>pytest</span>]);</span></span>
    in</span> {</span></span>
      packages</span>.</span>default</span> =</span> pkgs</span>.</span>writeShellApplication</span> {</span></span>
        name</span> =</span> "ttkv"</span>;</span></span>
        text</span> =</span> ''</span></span>
          ${</span>python</span>}/bin/python -mpytest ttkv_spec.py</span></span>
        ''</span>;</span></span>
      };</span></span>
    });</span></span>
}</span></span></code></pre>
To run the tests yourself, poke around in the code, etc. feel free to copy the
whole ttkv_py</code> directory from my programming workbench on
GitHub.</a></p>


A Time Traveling Key-Value Store
2019-04-16T00:00:00+00:00
Never Give Up</h1>
Once upon a time I interviewed for an ML/DS/software engineering position.
I didn’t get the gig (as the inimitable Time Hopper says, never give
up</a>), but came away
with a fun programming exercise.
In the room, I put together some reasonable Python to solve and test it, but I
wanted to try my hand at using
cats-effect</code></a> to work through this in
some functional Scala.</p>
What is Your Problem and Why is it Interesting?</h1>
Your mission, should you choose to accept it, is to implement (and test) a data
structure that looks a bit like a dictionary/map/hash/associative
array/whatever your language calls it, but with a twist.
In pseudo-Scala, your Time Traveling Key-Value Store</em> (TTKV from now on)
should have two methods that look a bit like the following (note that we change
this a little in our implementation for functional programming goodness):</p>
def</span> put</span>(</span>key</span>:</span> K</span>, </span>value</span>:</span> V</span>)</span>:</span> Unit</span></span>
def</span> get</span>(</span>key</span>:</span> K</span>, </span>time</span>:</span> Option</span>[</span>Long</span>]</span> =</span> None</span>)</span>:</span> Option</span>[</span>V</span>]</span></span></code></pre>
Calling put</code> should store the (key, value)</code> pair in your TTKV.
Calling get</code>, however, has some interesting behavior.
If you don’t supply an optional time</code>, it returns the latest value</code> you stored
with the key</code>.
If you do supply a time</code>, TTKV returns the value</code> it had for key</code> at that
time</code>.
For instance, if you first call TTKV.put("a", 1)</code> at time 3, then later at
time 7 you call TTKV.put("a", 2)</code>, then</p>

TTKV.get("a")</code> should be Some(2)</code>,</li>
TTKV.get("a", Some(6))</code> should be 1, since 6 is after 3 but before 7,</li>
TTKV.get("a", Some(n))</code> should be Some(2)</code> for any n ≥ 7</code></li>
TTKV.get("a", Some(n))</code> should be None</code> for any n < 3</code></li>
TTKV.get(x, y)</code> should be None</code> for any other x</code> and y</code>.</li>
</ul>
My Python in-the-room implementation relied on side-effects, reading a global
nanosecond
clock.</a>
This was less than ideal, and though it met the remit, it wasn’t easily
testable—which I suspect was the point of the exercise.</p>
By its nature, put</code> involves an effect: reading a time.
To do things functionally, I’ll turn to cats-effect</code> and explicitly make note
of this effectful computation.
This involves a slight change to TTKV’s API; while get</code> stays the same, we’ll
change put</code> to the following:</p>
def</span> put</span>[</span>F</span>[_]](</span>key</span>:</span> K</span>, </span>value</span>:</span> V</span>)</span></span>
             (</span>implicit</span> sync</span>:</span> Sync</span>[</span>F</span>], </span>clock</span>:</span> Clock</span>[</span>F</span>])</span>:</span> F</span>[</span>TTKV</span>[</span>K</span>, </span>V</span>]]</span></span></code></pre>
First, rather than mutating TTKV</code> in place, put</code> returns a new one with the
(key, value)</code> pair added for a specific time.
Second, we’re explicitly noting that this work takes place inside a Clock</code>
effect, from which we grab the current time.
I like how we can still get</code> the same way as before, since querying a TTKV
shouldn’t have to concern itself with effects, but this design specifically
highlights that put</code> is different.</p>
Implementation</h1>
Here’s my implementation in its entirety (again, head to
GitHub</a> to
see it in action):</p>
package</span> ttkv</span></span>
</span>
import</span> cats</span>.</span>effect</span>.{</span>Clock</span>, </span>Sync</span>}</span></span>
import</span> cats</span>.</span>implicits</span>.</span>_</span></span>
import</span> alleycats</span>.</span>std</span>.</span>iterable</span>.</span>_</span></span>
import</span> scala</span>.</span>collection</span>.</span>immutable</span>.</span>LongMap</span></span>
</span>
case class</span> TTKV</span>[</span>K</span>, </span>V</span>](</span>inner</span>:</span> Map</span>[</span>K</span>, </span>LongMap</span>[</span>V</span>]])</span> extends</span> AnyVal</span> {</span></span>
  def</span> put</span>[</span>F</span>[_]](</span>key</span>:</span> K</span>, </span>value</span>:</span> V</span>)</span></span>
               (</span>implicit</span> sync</span>:</span> Sync</span>[</span>F</span>])</span>:</span> F</span>[</span>TTKV</span>[</span>K</span>, </span>V</span>]]</span> =</span></span>
    Clock</span>[</span>F</span>].monotonic.map { t </span>=></span></span>
      val</span> m</span> =</span> inner.getOrElse(key, </span>LongMap</span>.empty)</span> +</span> (t.toNanos </span>-></span> value)</span></span>
      TTKV</span>(inner </span>+</span> (key </span>-></span> m))</span></span>
    }</span></span>
</span>
  def</span> get</span>(</span>key</span>:</span> K</span>, </span>time</span>:</span> Option</span>[</span>Long</span>]</span> =</span> None</span>)</span>:</span> Option</span>[</span>V</span>]</span> =</span></span>
    for</span> {</span></span>
      m </span><-</span> inner.get(key)</span></span>
      keys </span>=</span> m.keys.filter(_ </span><=</span> time.getOrElse(</span>Long</span>.</span>MaxValue</span>))</span></span>
      t0 </span><-</span> keys.maximumOption</span></span>
      value </span><-</span> m.get(t0)</span></span>
    }</span> yield</span> value</span></span>
</span>
  def</span> times</span>(</span>key</span>:</span> K</span>)</span>:</span> List</span>[</span>Long</span>]</span> =</span></span>
    inner.get(key).map(_.keys.toList.sorted).getOrElse(</span>Nil</span>)</span></span>
</span>
  def</span> times</span>:</span> List</span>[</span>Long</span>]</span> =</span></span>
    inner.keys.toList.flatMap(times).sorted</span></span>
}</span></span>
</span>
object</span> TTKV</span> {</span></span>
  def</span> empty</span>[</span>K</span>, </span>V</span>]</span>:</span> TTKV</span>[</span>K</span>, </span>V</span>]</span> =</span> TTKV</span>(</span>Map</span>.empty)</span></span>
}</span></span></code></pre>Will it Blend?</h1>
In the room, I wrote some simple unit tests for my Python implementation.
For this functional version, I put together some property-based tests with
ScalaCheck</code>.</a>
You can find the entirety on
GitHub</a>; I’ll
copy some highlights here.</p>
First off, calling get</code> on an empty TTKV should return None</code>:</p>
property(</span>"initially empty"</span>)</span> =</span> forAll {</span></span>
  (</span>a</span>:</span> Char</span>)</span> =></span></span>
    TTKV</span>.empty.get(a)</span> ==</span> None</span></span>
}</span></span></code></pre>
Similarly, calling get</code> immediately after a put</code> should return the same thing
you put</code>:</p>
property(</span>"single get"</span>)</span> =</span> forAll {</span></span>
  (</span>a</span>:</span> Char</span>, </span>x</span>:</span> Int</span>)</span> =></span></span>
    TTKV</span>.empty.put(a, x).map(_.get(a)</span> ==</span> Some</span>(x)).unsafeRunSync</span></span>
}</span></span></code></pre>
More importantly, calling get</code> with any time prior to when you put</code> a thing
should return None</code>:</p>
property(</span>"get before time"</span>)</span> =</span> forAll {</span></span>
  (</span>a</span>:</span> Char</span>, </span>x</span>:</span> Int</span>, </span>t</span>:</span> Long</span>)</span> =></span></span>
    (t </span>></span> 0</span>)</span> ==></span></span>
      TTKV</span>.empty</span></span>
        .put(a, x)</span></span>
        .map { m </span>=></span></span>
          val</span> t0</span> =</span> m.times(a).head</span></span>
          m.get(a, </span>Some</span>(t0 </span>-</span> t))</span> ==</span> None</span></span>
        }.unsafeRunSync</span></span>
}</span></span></code></pre>
And for two puts</code> with the same key, calling get</code> with any time between</em> the
two times should return the value from the first</em> put</code>:</p>
property(</span>"middle get"</span>)</span> =</span> forAll {</span></span>
  (</span>a</span>:</span> Char</span>, </span>x</span>:</span> Int</span>, </span>y</span>:</span> Int</span>, </span>d</span>:</span> Double</span>)</span> =></span></span>
    ((</span>0</span> <=</span> d)</span> &&</span> (d </span><</span> 1</span>))</span> ==></span></span>
      TTKV</span>.empty</span></span>
        .put(a, x)</span></span>
        .flatMap(_.put(a, y))</span></span>
        .map { m </span>=></span></span>
          val</span> ts</span> =</span> m.times(a)</span></span>
          val</span> t0</span> =</span> ts.head</span></span>
          val</span> t1</span> =</span> ts.last</span></span>
          val</span> δ</span> =</span> (t1 </span>-</span> t0)</span> *</span> d</span></span>
          m.get(a, </span>Some</span>(t0 </span>+</span> δ.toLong))</span> ==</span> Some</span>(x)</span></span>
        }</span></span>
        .unsafeRunSync</span></span>
}</span></span></code></pre>Appendix</h1>
Here’s the flake.nix</code> file to specify the W O R L D:</p>
{</span></span>
  description</span> =</span> "Scala TTKV"</span>;</span></span>
</span>
  inputs</span> =</span> {</span></span>
    flake-utils</span>.</span>url</span> =</span> "github:numtide/flake-utils"</span>;</span></span>
    nixpkgs</span>.</span>url</span> =</span> "github:NixOS/nixpkgs/nixos-23.11"</span>;</span></span>
  };</span></span>
</span>
  outputs</span> =</span> {</span></span>
    self</span>,</span></span>
    flake-utils</span>,</span></span>
    nixpkgs</span>,</span></span>
  }:</span></span>
    flake-utils</span>.</span>lib</span>.</span>eachDefaultSystem</span> (system:</span> let</span></span>
      pkgs</span> =</span> import</span> nixpkgs</span> {</span>inherit</span> system</span>;};</span></span>
    in</span> {</span></span>
      packages</span>.</span>default</span> =</span> pkgs</span>.</span>writeShellApplication</span> {</span></span>
        name</span> =</span> "ttkv"</span>;</span></span>
        runtimeInputs</span> =</span> [</span>pkgs</span>.</span>sbt</span>];</span></span>
        text</span> =</span> ''</span></span>
          ${</span>pkgs</span>.</span>sbt</span>}/bin/sbt test</span></span>
        ''</span>;</span></span>
      };</span></span>
    });</span></span>
}</span></span></code></pre>
The build.sbt</code> is similarly minimal:</p>
lazy val</span> root</span> =</span> (project in file(</span>"."</span>))</span></span>
  .settings(</span></span>
    name </span>:=</span> "ttkv"</span>,</span></span>
    libraryDependencies </span>++=</span> Seq</span>(</span></span>
      "org.typelevel"</span> %%</span> "cats-effect"</span> %</span> "3.5.4"</span>,</span></span>
      "org.typelevel"</span> %%</span> "alleycats-core"</span> %</span> "2.10.0"</span>,</span></span>
      "org.scalacheck"</span> %%</span> "scalacheck"</span> %</span> "1.15.4"</span> %</span> "test"</span></span>
    ),</span></span>
    scalacOptions </span>++=</span> Seq</span>(</span></span>
      "-feature"</span>,</span></span>
      "-deprecation"</span>,</span></span>
      "-unchecked"</span>,</span></span>
      "-language:higherKinds"</span>,</span></span>
      "-Ypartial-unification"</span></span>
    )</span></span>
  )</span></span></code></pre>
To run the tests, poke around in the code, etc. feel free to copy the whole
ttkv</code> directory from my programming workbench on
GitHub.</a></p>


Slides: Algebraic Structure, Computational Benefits
2018-02-20T00:00:00+00:00
Algebraic Structure, Computational Benefits</h1>
I recently gave a talk at work titled “Algebraic Structure, Computational
Benefits.”
In it, I defined a few basic (as in fundamental, not as in easy) structures
like semigroups, monoids, and semirings, and I discussed the computational
benefits one can derive by making these structures explicit in code, like:</p>

the most generic of generic algorithmic code</li>
parallelism for free, a.k.a. associativity implies Map-Reduce</code></li>
associativity + commutativity ⇒ Map-Shuffle-Reduce</code></li>
</ul>
I also talked a little bit about one of my recent favorites, the
Tropical Semiring</em></a>, and how
it turns shortest-distance problems in graph theory into iterated linear
algebra.</p>
Should any of this spark your interest, slides are available here.</a></p>


Fourier Transform in Clojure, Reproducibly with Nix
2017-12-12T00:00:00+00:00
Combining previous work with new things</h1>
In a previous post</a>,
we explored an example from Persi Diaconis using the Fourier Transform to
change a collection of permutations into a table where the (i, j)</em> entry is
the number of voters who ranked candidate i</em> in place j</em>; this
transformation consists of first mapping each permutation to its matrix
representation, then adding the matrices together.</p>
There’s more to explore here, however; since counts are just integers and
addition of integers is both associative and commutative, this matrix addition
is also commutative—that is, we can reorder and group the summands as much as
we like.
Thus, this problem is well suited to map-reduce-like approaches, since
associativity implies parallelism.
With commutativity, we can include a “shuffle” step, too.
This is what the Clojure</code> library tesser</code></a>
provides.
Combined with core.matrix</code></a> for
matrices, tesser</code> allows us to</p>

map the vote vectors to their permutation matrices</li>
add them together</li>
</ol>
but, contrasted with core Clojure</code>’s map</code> and reduce</code>, we can take advantage
of the mathematical structure to do all of step 1 in parallel, then all of
step 2 in parallel with arbitrary shuffling.</p>
Clojure</code> Code</h1>
Here’s Clojure</code> code that, after simulating a collection of votes, calculates
the desired table in two different ways:</p>

in serial, with the default map</code> and reduce</code>, and</li>
using tesser</code> to take full advantage of associativity and commutativity.</li>
</ol>
It then checks that the two answers (imaginatively called a</code> and b</code>) are
equal, then prints out a</code>.</p>
(</span>require</span> '[tesser.core</span> :as</span> t])</span></span>
(</span>require</span> '[clojure.core.matrix</span> :as</span> m])</span></span>
</span>
(</span>let</span></span>
  [k</span> 7</span></span>
   n</span> 10000</span></span>
   zero (</span>m/zero-matrix</span> k k)</span></span>
   vecs (</span>for</span> [_ (</span>range</span> n)] (</span>shuffle</span> (</span>range</span> k)))</span></span>
   a (</span>->></span> vecs</span></span>
          (</span>map</span> m/permutation-matrix)</span></span>
          (</span>reduce</span> m/add zero))</span></span>
   b (</span>->></span> (</span>t/map</span> m/permutation-matrix)</span></span>
          (</span>t/reduce</span> m/add zero)</span></span>
          (</span>t/tesser</span> [vecs]))</span></span>
   matrices-equal (</span>if</span> (</span>=</span> a b)</span> "Yes" "No"</span>)]</span></span>
  (</span>println</span> (</span>str</span> "</span>\n\n</span>Are a and b equal? "</span> matrices-equal</span> "</span>\n</span>"</span>))</span></span>
  (</span>m/pm</span> a))</span></span></code></pre>
Which, on one run, gave the following output:</p>
Are a and b equal? Yes</span></span>
</span>
[[1448.000 1419.000 1434.000 1505.000 1444.000 1389.000 1361.000]</span></span>
 [1428.000 1429.000 1430.000 1404.000 1384.000 1475.000 1450.000]</span></span>
 [1466.000 1471.000 1367.000 1458.000 1443.000 1408.000 1387.000]</span></span>
 [1427.000 1369.000 1434.000 1429.000 1452.000 1434.000 1455.000]</span></span>
 [1393.000 1388.000 1481.000 1444.000 1380.000 1439.000 1475.000]</span></span>
 [1412.000 1429.000 1447.000 1389.000 1415.000 1486.000 1422.000]</span></span>
 [1426.000 1495.000 1407.000 1371.000 1482.000 1369.000 1450.000]]</span></span></code></pre>Nix</code> Expressions</h1>
Code demos are fun, but too often they fall victim to bit-rot and stop working.
To combat this, we can strive for reproducibility, an important goal of data
science</a>.
One tool that can help is nix</code>.</p>
Nix</code></a> is a package manager, like homebrew</code> for OS X</code>,
that, through a novel approach, goes a very</em> long way towards reproducibility.
By specifying everything in a declarative, functional language, we can be sure
we’re building the same thing twice.
With nix</code>, we can specify the exact</em> versions of tesser</code> and core.matrix</code>
we used, along with how to build them.
Then, we can provide a function to reproducibly run the above example.
The nix</code> expression to do so is rather long—though that’s a small price to
pay for the assurance we get.</p>
Here’s the content of flake.nix</code>.
Save this in a file called flake.nix</code>, and save the above Clojure</code> code in diaconis.clj</code> in the same directory.
Then run nix run</code> to (eventually) see the results.</p>
{</span></span>
  description</span> =</span> "Fourier Transform in Clojure"</span>;</span></span>
</span>
  inputs</span> =</span> {</span></span>
    flake-utils</span>.</span>url</span> =</span> "github:numtide/flake-utils"</span>;</span></span>
    nixpkgs</span>.</span>url</span> =</span> "github:NixOS/nixpkgs/nixos-22.11"</span>;</span></span>
  };</span></span>
</span>
  outputs</span> =</span> {</span></span>
    self</span>,</span></span>
    flake-utils</span>,</span></span>
    nixpkgs</span>,</span></span>
  }:</span></span>
    flake-utils</span>.</span>lib</span>.</span>eachDefaultSystem</span> (system:</span> let</span></span>
      pkgs</span> =</span> import</span> nixpkgs</span> {</span>inherit</span> system</span>;};</span></span>
      leinFromGitHub</span> =</span> {</span></span>
        name</span>,</span></span>
        owner</span>,</span></span>
        repo</span>,</span></span>
        rev</span>,</span></span>
        sha256</span>,</span></span>
        cd</span> ?</span> "."</span>,</span></span>
      }:</span></span>
        pkgs</span>.</span>stdenv</span>.</span>mkDerivation</span> {</span></span>
          name</span> =</span> name</span>;</span></span>
          src</span> =</span> pkgs</span>.</span>fetchFromGitHub</span> {</span></span>
            owner</span> =</span> owner</span>;</span></span>
            repo</span> =</span> repo</span>;</span></span>
            rev</span> =</span> rev</span>;</span></span>
            sha256</span> =</span> sha256</span>;</span></span>
          };</span></span>
          buildInputs</span> =</span> [</span>pkgs</span>.</span>leiningen</span>];</span></span>
          # https://blog.jeaye.com/2017/07/30/nixos-revisited/</span></span>
          buildPhase</span> =</span> ''</span></span>
            export LEIN_HOME=$PWD/.lein</span></span>
            mkdir -p $LEIN_HOME</span></span>
            echo "{:user {:local-repo \"$LEIN_HOME\"}}" > $LEIN_HOME/profiles.clj</span></span>
            cd ${</span>cd</span>}</span></span>
            LEIN_SNAPSHOTS_IN_RELEASE=1 ${</span>pkgs</span>.</span>leiningen</span>}/bin/lein uberjar</span></span>
          ''</span>;</span></span>
          installPhase</span> =</span> ''</span></span>
            cp target/${</span>repo</span>}*-standalone.jar $out</span></span>
          ''</span>;</span></span>
        };</span></span>
      tesser</span> =</span> leinFromGitHub</span> {</span></span>
        name</span> =</span> "tesser"</span>;</span></span>
        owner</span> =</span> "aphyr"</span>;</span></span>
        repo</span> =</span> "tesser"</span>;</span></span>
        rev</span> =</span> "d82895390264d8f6bf012fa4053a2003a500b574"</span>;</span> # release 1.0.4</span></span>
        sha256</span> =</span> "0k2shy6ccig00lc9y6y8lyh7bnd250xxp5nh8fz254f2kq8ib4mn"</span>;</span></span>
        cd</span> =</span> "core"</span>;</span></span>
      };</span></span>
      matrix</span> =</span> leinFromGitHub</span> {</span></span>
        name</span> =</span> "matrix"</span>;</span></span>
        owner</span> =</span> "mikera"</span>;</span></span>
        repo</span> =</span> "core.matrix"</span>;</span></span>
        rev</span> =</span> "07cb88b1b43ccc07f3f7e8634e1eccdb6986049b"</span>;</span> # develop branch as of 2021-11-19</span></span>
        sha256</span> =</span> "1dn5hy5mdgl2bcmav8qll3qxqbscb340c462kkj728sfpk4ch5xd"</span>;</span></span>
      };</span></span>
    in</span> {</span></span>
      packages</span>.</span>default</span> =</span> pkgs</span>.</span>writeShellApplication</span> rec</span> {</span></span>
        name</span> =</span> "diaconis.clj"</span>;</span></span>
        runtimeInputs</span> =</span> [</span>pkgs</span>.</span>clojure pkgs</span>.</span>jdk</span>];</span></span>
        text</span> =</span> ''</span></span>
          java -cp ${</span>tesser</span>}:${</span>matrix</span>} clojure.main ${</span>name</span>}</span></span>
        ''</span>;</span></span>
      };</span></span>
    });</span></span>
}</span></span>
</span></code></pre>


Two Short Python Things
2017-09-05T00:00:00+00:00
Sorry</h1>
Apologies, as ever, for taking so</em> incredibly long between posts.
That said if the inimitable Roger Peng can get away with letting some time
lapse while working on a project, perhaps you can forgive me?</p>

Sometimes it takes a while for a project to get going.</p>
— Roger D. Peng (@rdpeng) August 24, 2017, Twitter</p>
</blockquote>
Back to Python</h1>
After a long time in the Scala wilderness, I’ve recently had a chance to write
some Python again for scripting purposes at work.
I had a bit of freedom with regards to process, so I explicitly chose Python
3.6.
I’m here to tell you that Python 3.6 is really nice and to exhibit two things
that made my life much better.</p>
What’s up, __doc__</code>?</h1>
Especially for scripts, I’ve gotten into the habit of putting all of the actual
work and logic of my Python program in functions (usually a main</code> function
calling other steps/etc.).
I then have an argparse.ArgumentParser</code> at the bottom (wrapped in an if __name__ == '__main__'</code> scripting
statement</a>) that parses
command line arguments and passes them to the main</code> function, like so:</p>
if</span> __name__</span> ==</span> '__main__'</span>:</span></span>
    P</span> =</span> ArgumentParser()</span></span>
    P.add_argument(…)</span></span>
    …</span></span>
    main(P.parse_args())</span></span></code></pre>
Given this setup, one can pass the -h</code> or --help</code> flag to the script and
receive some information about the different arguments you’ve added to the
parser.
However, it’s good form to add a description</code> to your ArgumentParser</code> that
explains the purpose and usage of the script.</p>
If you’re following best practices (e.g. by trying to appease
pylint</code></a>), you’ve already added a docstring to the
top of your file that explains its purpose and usage.
It can be a pain (or at least problematic) to keep this docstring and the
parser’s description</code> in sync.</p>
I recently stumbled across a nice solution to this problem: only write the
docstring, and then use the __doc__</code> magic
variable</a> as your description</code> like so:</p>
if</span> __name__</span> ==</span> '__main__'</span>:</span></span>
    P</span> =</span> ArgumentParser(</span>description</span>=</span>__doc__</span>)</span></span></code></pre>
This approach does limit the length and expressiveness you can fit into the
docstring/description due to the constraints of printing a help message, but
I haven’t found it to be a problem since my script docstrings tend to be
exactly what I’d want to print a help message anyway.</p>
typing.NamedTuple</code> is my new favorite</h1>
Even more mindbending is the new (in Python 3.5) typing module</code></a>.
Coupled with the mypy</code> static checker</a> and type
hints</a>,
this allows you to write Python with static types and check them prior</em> to
running code in production.
For instance, the main</code> function for my scripts that once looked like this:</p>
def</span> main</span>(args):</span></span>
    """Performs the main work</span></span>
</span>
    Parameters</span></span>
    ----------</span></span>
    args: Namespace</span></span>
        Arguments passed on the command line, specifying what to do</span></span>
</span>
    Returns</span></span>
    -------</span></span>
    None</span></span>
    """</span></span>
    # stuff and things</span></span>
    …</span></span></code></pre>
now looks like this:</p>
def</span> main</span>(args: Namespace) -></span> None</span>:</span></span>
    """Performs the main work</span></span>
</span>
    Parameters</span></span>
    ----------</span></span>
    args: Arguments passed on the command line, specifying what to do</span></span>
    """</span></span>
    # stuff and things</span></span>
    …</span></span></code></pre>
Some folks might cry that this doesn’t look like Python anymore, but to them I
counter</p>

Python Enhancement Proposal 484</a>
already accepted type hints into the language, and</li>
if you’re writing suitably descriptive
docstrings</a> for your functions,
you’re already</em> writing the input and output types anyway; why not statically
check them, too?</li>
</ul>
If I haven’t lost you with the type hinting above, I have something else to
show you: the new NamedTuple</code>
class.</a>
When writing Python classes, I often get annoyed with having to specify the
typical initialization code one finds in the __init__</code> method, like</p>
class</span> MyThing</span>:</span></span>
</span>
    def</span> __init__</span>(self, a, b, c, d):</span></span>
        self</span>.a</span> =</span> a</span>  # this is an integer</span></span>
        self</span>.b</span> =</span> b</span>  # this is a float</span></span>
        self</span>.c</span> =</span> c</span>  # this is a string</span></span>
        self</span>.d</span> =</span> d</span>  # this is a dictionary</span></span>
</span>
    … </span># other methods here</span></span></code></pre>
Unless I need</em> some fancy initialization logic, I’d prefer to have this
attribute setup taken care of by default; enter NamedTuple</code>:</p>
class</span> MyThing</span>(</span>NamedTuple</span>):</span></span>
    a:</span> int</span></span>
    b:</span> float</span></span>
    c:</span> str</span></span>
    d:</span> dict</span></span>
</span>
    … </span># other methods here</span></span></code></pre>
Besides being shorter (“brevity facilitates reasoning,” as one of my favorite
papers</a> explains), this version
explicitly</em> lists the attribute types in a way checkable via mypy</code>.
For instance, x = MyThing(1, 2, 'three', {'four': 4})</code> is just fine, but
if you try to sneak y = MyThing(1.0, 2, 'three', {'four': 4})</code> past mypy</code>
it’ll complain</p>
error: Argument 1 to "MyThing" has incompatible type "float"; expected "int"</span></span></code></pre>
Perhaps my brain has rotted away after relying on
statically</a> typed</a>
languages</a>, but I’m a fan.</p>


Cross Compiling Static Rust Binaries for Linux on OS X
2016-09-26T00:00:00+00:00
Recently I wrote a small Rust</code></a> executable for work.
I wrote it on my Apple laptop, but wanted to run it on our Linux servers.
This note hopes to document how I did it so that I and others may repeat my
experience.</p>
Note: I only managed to pull this off because of the helpful folks on the
#rust</code> and #rust-internals</code> IRC channels. Thanks everyone!</em></p>
Set up</h1>
There are plenty of tutorials out there for getting started with Rust</code>, though
it’s tough to beat the official
book</a>.
In order to get a working environment going, I recommend using
rustup</a>.</p>
First, create a new binary project with cargo</code>:</p>
cargo</span> init</span> --bin</span> my_proj</span></span>
cd</span> my_proj</span></span></code></pre>
In order to get stuff working later, use the nightly</code> branch of Rust</code>:</p>
rustup</span> override set nightly</span></span></code></pre>
Write your project as usual, editing src/main.rs</code> to your heart’s content.</p>
musl</code></h1>
In order to get your executable to run on the Linux server, you’ll need to
compile & link it for that platform.
I’ve had luck creating an entirely</em> static executable using musl</code>, a rewrite
of the C</code> standard library intended for smaller, embedded targets.
To get the necessary tools installed on OS X, use homebrew</code> with this helpful
tap</a>.</p>
Once you’ve got the linker & other compilation tools installed (as the above
link says, that’ll take a while), you’ll need to let cargo</code> know to use them
when compiling for the Linux target.
Create a subdirectory to the my_proj</code> directory called .cargo</code>; then in
.cargo/config</code>, write the following</p>
[</span>target</span>.</span>x86_64-unknown-linux-musl</span>]</span></span>
linker =</span> "x86_64-linux-musl-gcc"</span></span></code></pre>
You’ll also need to have rustup</code> specify the target:</p>
rustup</span> target add x86_64-unknown-linux-musl</span></span></code></pre>Compile & Deploy</h1>
To compile your executable for the server, run</p>
cargo</span> build</span> --release --target=x86_64-unknown-linux-musl</span></span></code></pre>
Copy it to the server and you’re good to go:</p>
scp</span> target/x86_64-unknown-linux-musl/release/my_proj ⟨your_server_name⟩:</span></span></code></pre>UPX (added 2016-10-17)</h1>
If you’re concerned about the binary
size</a>
of your new executable, check out UPX</code></a>.
After installing it on my laptop via brew install upx</code>, I ran upx -9</code> on an
executable created with the above instructions.
While the executable was an overly simplistic example, upx</code> compressed it down
to 34%</em> of the original size.
Even if you don’t care about the size of the binary once it’s on the server, it
at least made the scp</code> go faster.</p>


ScalaCheck and the Unkillable Trash
2016-04-21T00:00:00+00:00
Well that’s weird</h1>
I ran into an interesting problem in a recent
Scala</code></a> project.
Let’s set the scene …</p>
Quickly Checking in Scala</h1>
In the project in question, I reached for a
library</a> to do a fair amount of JSON
parsing.
I hadn’t used the library before, so I ended up writing some
ScalaCheck</code></a> tests for the convenience/wrapper
functions I made.</p>
One test involved writing data to a temporary file, reading and parsing that
information, then comparing the parsed results to the original data.
If all went well, the parsed and original data should match perfectly.
Either way, there was clean up code at the end of the test to delete the
temporary file.
My mistake was in letting ScalaCheck</code> generate strings of arbitrary characters
for the temporary file’s name, rather than restricting those filenames to
printable or alphanumeric characters only.</p>
Obscure OS X obstacles</h1>
It turns out that the latest version of Mac OS X can’t delete—or
otherwise get a handle on, via GUI or CLI—files with certain characters
in the
name</a>.
So, when ScalaCheck</code> created such a temporary file, my clean up code was
unable to rm</code> it.
I tried everything</em> the Quantifind Dev team</a> and I
could think of, to no avail.
I now have, tucked away in a directory called unkillable_trash</code>, a file that
nothing short of reformatting my hard drive will remove.</p>
Update (2017-04-13)</h1>
Since the time of this post, a botched update necessitated reinstalling the OS
on my work computer; the unkillable_trash</code> is, sadly, now gone.</p>


New Job, Hire in DC, and a Short Revisit of Finite Fields
2016-04-07T00:00:00+00:00
Hola</h1>
Is it poor form to start each post with an apology for the time elapsed since
the previous one?
Either way, hello dear reader.
Beyond being the father of a rambunctious young’un, things have changed around
here since we last spoke.</p>
New Job, Plus Why Tech Companies Should Hire in DC</h1>
In January I started a new gig as a Data Scientist for
Quantifind</a>.
I’m excited about the opportunity, and the balance of faster-paced demands and
more blue-sky research has been great.</p>
This new job has highlighted something for me: if you’re a tech/data/whatever
company, you should hire people from the D.C. area.
There are a large number of people here who have the technical chops to do data
and software work, and who—given
agency</a>
reorganizations</a>
and general low
morale</a>—are
probably open to new opportunities.
While some may be able to pick up and move to Silicon Valley, I’d wager that
many are (like myself) unable to uproot their lives and families.
I’m fortunate that Quantifind hired me to work in the D.C. office, even though
they’re based in California.
Other companies should consider tapping the large pool of talent we have in the
region.</p>
Revisiting Finite Fields</h1>
Now a short technical note.
In a previous post</a>, I talked about
my difficulties in working with finite fields in software.
I’m starting to think that for parameterized mathematical objects (e.g. by a
prime of a prime and a polynomial, like finite fields) might be good candidates
for a dependently</a>
typed</a>
language like Idris</code></a>.
I don’t have any experience with Idris</code> myself, but I’d be curious to see a
finite field implementation in it.</p>


Learning By Building: A Simple Neural Network Implementation
2015-07-20T00:00:00+00:00
Small baby, no sleep make Graham a dull blogger</h1>
Apologies, readers, for my long absence.
My family grew this last year, so I’ve had less time for writing.</p>
Feynman’s Blackboard</h1>
On Richard Feynman’s
blackboard</a> at the
time of his death was the
quote</a> “What I cannot create, I
do not understand.”</em>
Powerful words, and I definitely think there’s some truth to them.</p>
Recently I came across a popular, succinct
post</a> on
implementing a neural
network</a> in Python.
I realized that although I’ve used them before and had a cursory understanding
of how they worked under the hood, I didn’t really grok</em> them; I had always
turned to a library.
Admittedly, turning to a well-developed library is exactly what one should do
in production, but for my own edification I decided to put together a Python
implementation</a>
of my own.
This implementation is rather simplistic, but it has three features I like:</p>

It’s simple enough that someone as sleep deprived as myself can
understand it.</li>
It feels Pythonic; it relies only upon NumPy</code> and has an API similar to
sklearn</code>’s classifiers</li>
It can handle an arbitrary number of hidden layers of arbitrary size.</li>
</ol>
Lesson Learned</h1>
As awesome as neural nets are, as many varieties and variations there are, and
as amazing as some of the
applications</a>
may be, I discovered that at their base, they’re rather simple.1</a></sup>
Neural networks boil down to iterated linear algebra—which totally makes
sense, since they’re being implemented on GPUs all over the place.
Though I wouldn’t recommend going back to basics all the time (especially if
you’re on a deadline), I’m definitely glad I took the time to work this
through.</p>



The vanilla version is, anyway; other variations are admittedly more complex. ↩</a></p>
</li>
</ol>
</section>


Predicting Transformer Affiliation from Motto
2014-10-26T00:00:00+00:00
The Story</h1>
To give an example of working with data while still having fun, I’ve decided to
tackle Transformers</a>.
Specifically, after finding this
dataset</a> someone scraped
together from Wikipedia, I thought it’d be interesting to see if we can
predict the affiliation (Autobot vs. Decepticon) of a character using one of
Scikit-Learn</a>’s
naive Bayes
classifiers</a>.</p>
Enough chit-chat; let’s roll up our sleeves and dive in!</p>
Preliminaries</h1>
Checking out the data, it looks like the entries are indexed by name (on the
far left of the line), which approximates the Transformer’s name.
Let’s see how many entries we’re dealing with:</p>
$</span> grep</span> -P</span> '^\S' infobox_transformers_character.yaml</span> |</span> wc</span> -l</span></span>
718</span></span></code></pre>
There are 718 entries, but it looks like there may be repeats (i.e., either the
data isn’t as clean as we’d hoped, or the names are not a unique enough
identifier).</p>
$</span> grep</span> -P</span> '^\S' infobox_transformers_character.yaml</span> |</span>\</span></span>
      uniq</span> -c</span> |</span>\</span></span>
      awk</span> '{print $1}'</span> |</span>\</span></span>
      sort</span> -n</span> |</span>\</span></span>
      uniq</span> -c</span></span>
250</span>  1</span></span>
 85</span>  2</span></span>
 45</span>  3</span></span>
 19</span>  4</span></span>
 12</span>  5</span></span>
  2</span>  6</span></span>
  1</span>  7</span></span>
  1</span>  8</span></span></code></pre>
So it looks like the majority of names aren’t repeated, but there are a few;
one is even repeated 8 times!</p>
$</span> grep</span> -P</span> '^\S' infobox_transformers_character.yaml</span> |</span>\</span></span>
      uniq</span> -c</span> |</span>\</span></span>
      sort</span> -nr</span> |</span>\</span></span>
      head</span></span>
8</span> Prowl_</span>(</span>Transformers</span>)</span>:</span></span>
7</span> Bonecrusher_</span>(</span>Transformers</span>)</span>:</span></span>
6</span> Ultra_Magnus:</span></span>
6</span> Ironhide:</span></span>
5</span> Thrust_</span>(</span>Transformers</span>)</span>:</span></span>
5</span> Swindle_</span>(</span>Transformers</span>)</span>:</span></span>
5</span> Snarl_</span>(</span>Transformers</span>)</span>:</span></span>
5</span> Skydive_</span>(</span>Transformers</span>)</span>:</span></span>
5</span> Scavenger_</span>(</span>Transformers</span>)</span>:</span></span>
5</span> Razorclaw:</span></span></code></pre>
Prowl is the most popular, eh? Let’s look at the different entries with that
name.</p>
$</span> grep</span> -P -A5</span> '^Prowl' infobox_transformers_character.yaml</span></span>
# output omitted</span></span></code></pre>
So these certainly seem to be unique entries; we should uniquify these names,
because PyYAML</code> turns YAML into a dictionary (and therefore squashes things
under a single heading).</p>
$</span> cat infobox_transformers_character.yaml</span> |</span>\</span></span>
    perl</span> -ple</span> 'BEGIN{$n = 0;} if (m/(^\S+)[:]/) { $_ = $1 . ($n++) . ":"; }'</span></span>
# output omitted</span></span></code></pre>
Looks good; let’s save it to a new YAML file.</p>
$</span> perl</span> -ple</span> 'BEGIN{$n = 0;} if (m/(^\S+)[:]/) { $_ = $1 . ($n++) . ":"; }'</span> \</span></span>
    infobox_transformers_character.yaml</span> ></span> unique_names.yaml</span></span></code></pre>Bring out the Python</h1>
Let’s start up IPython and start digging.</p>
>>> import</span> yaml</span></span>
>>> with</span> open</span>(</span>'unique_names.yaml'</span>)</span> as</span> f:</span></span>
        raw_data</span> =</span> yaml.load(f)</span></span></code></pre>
Loading that raw data into Pandas</code>:</p>
>>> import</span> numpy</span> as</span> np</span></span>
>>> import</span> pandas</span> as</span> pd</span></span>
>>></span> messy_df</span> =</span> pd.DataFrame(raw_data).transpose()</span></span>
>>></span> has_motto</span> =</span> messy_df[messy_df.motto.notnull()]</span></span>
>>></span> len</span>(has_motto)</span></span>
485</span></span>
>></span> any</span>(has_motto.affiliation.isnull())</span></span>
False</span></span></code></pre>
There are 485 entries that have a motto, and every entry with a motto also has
an affiliation.
As you might expect, though, our data isn’t clean enough yet; there are some
affiliations we don’t expect (i.e. not just “Autobot” or “Decepticon”), and the
mottos aren’t in the cleanest form.</p>
>>></span> has_motto.affiliation.unique()</span></span>
array([</span>'Decepticon'</span>,</span> 'Autobot'</span>,</span> 'Maximal'</span>,</span> 'Predacon'</span>,</span></span>
       'Autobot, later Maximal'</span>,</span> 'Mini-Con'</span>,</span></span>
       'Predacon, later Maximal then Decepticon'</span>,</span></span>
       'Maximal, later Autobot'</span>,</span> 'Vehicon'</span>,</span></span>
       'Maximal, </span>\\</span>n Former Predacon'</span>,</span> 'Mutant'</span>,</span></span>
       'Decepticon, later Autobot'</span>,</span> 'None/Decepticon'</span>,</span></span>
       'Decepticon, later Predacon'</span>,</span> 'botconcron'</span>],</span></span>
       dtype</span>=</span>object</span>)</span></span>
>>></span> has_motto.affiliation.value_counts()</span></span>
Autobot</span>                                    213</span></span>
Decepticon</span>                                 143</span></span>
Mini</span>-</span>Con</span>                                    40</span></span>
Maximal</span>                                     28</span></span>
Predacon</span>                                    23</span></span>
Vehicon</span>                                      9</span></span>
Maximal, later Autobot</span>                       7</span></span>
Decepticon, later Autobot</span>                    6</span></span>
Mutant</span>                                       4</span></span>
Autobot, later Maximal</span>                       4</span></span>
Maximal, \</span>n Former Predacon                  3</span></span>
None</span>/</span>Decepticon</span>                              2</span></span>
Decepticon, later Predacon</span>                   1</span></span>
botconcron</span>                                   1</span></span>
Predacon, later Maximal then Decepticon</span>      1</span></span></code></pre>
Consulting some expert knowledge</a>, it looks like</p>

Maximals can count as Autobots since they’re descended from them</li>
Predacons can count as Decepticons, since they’re the enemies of the Maximals</li>
Mini-Cons can be either, so we’ll have to ignore them</li>
Vehicons are Decepticons</li>
I was unable to find botconcron; this looks like a transcription error, but
there’s only one so we’ll drop it</li>
</ul>
Cleaning up and creating a new variable numeric_affil</code> with 0 = Decepticon and
1 = Autobot:</p>
>>> def</span> normalize_aff</span>(aff):</span></span>
        if</span> any</span>(bad</span> in</span> aff</span> for</span> bad</span> in</span> 'Decepticon Predacon Vehicon'</span>.split()):</span></span>
            return</span> 0</span></span>
        elif</span> any</span>(good</span> in</span> aff</span> for</span> good</span> in</span> 'Autobot Maximal'</span>.split()):</span></span>
            return</span> 1</span></span>
        else</span>:</span></span>
            return</span> np.nan</span></span>
</span>
>>></span> has_motto[</span>'numeric_affil'</span>]</span> =</span> has_motto.affiliation.apply(normalize_aff)</span></span></code></pre>
Cleaning the mottos:</p>
>>> def</span> clean_motto</span>(m):</span></span>
        return</span> m.lower()</span></span>
                .replace(</span>'</span>\u2019</span> '</span>,</span> "'"</span>)</span></span>
                .replace(</span>'</span>\u2019</span>'</span>,</span> "'"</span>)</span></span>
                .replace(</span>'</span>\\</span>"'</span>,</span> ''</span>)</span></span>
                .replace(</span>'<br>'</span>,</span> ''</span>)</span></span>
                .replace(</span>'</br>'</span>,</span> ''</span>)</span></span>
>>></span> has_motto[</span>'clean_motto'</span>]</span> =</span> has_motto.motto.apply(clean_motto)</span></span></code></pre>
This still isn’t perfect, but it’s much better than we started with.</p>
Now, we’ve got our cleaned data:</p>
>>></span> data</span> =</span> has_motto.dropna()[[</span>'numeric_affil'</span>,</span> 'clean_motto'</span>]]</span></span>
>>></span> data.columns</span> =</span> 'affiliation motto'</span>.split()</span></span></code></pre>Scikit-Learn</h1>
Let’s import what we’ll need: two vectorizers for transforming text into
matrices, and the two naive Bayes classifiers we’ll compare.
We’ll also grab the word_tokenize</code> function from NLTK</code>.</p>
>>> from</span> nltk</span> import</span> word_tokenize</span></span>
>>> from</span> sklearn.feature_extraction.text</span> import</span> CountVectorizer</span></span>
>>> from</span> sklearn.feature_extraction.text</span> import</span> TfidfVectorizer</span></span>
>>> from</span> sklearn.naive_bayes</span> import</span> BernoulliNB</span></span>
>>> from</span> sklearn.naive_bayes</span> import</span>  GaussianNB</span></span>
>>></span> binary_vectorizer</span> =</span> CountVectorizer(</span></span>
        binary</span>=</span>True</span>,</span> min_df</span>=</span>1</span>,</span> tokenizer</span>=</span>word_tokenize)</span></span>
>>></span> tfidf_vectorizer</span> =</span> TfidfVectorizer(</span></span>
        min_df</span>=</span>1</span>,</span> tokenizer</span>=</span>word_tokenize)</span></span>
>>></span> binary_nb, gaussian_nb</span> =</span> BernoulliNB(), GaussianNB()</span></span></code></pre>
To compare the Bernoulli and the Gaussian naive Bayes classifiers, we will need
to use our vectorizers on them both.
To play fairly, we’ll break out data into train and test groups.</p>
>>> from</span> sklearn.cross_validation</span> import</span> train_test_split</span></span>
>>></span> binary_motto</span> =</span> binary_vectorizer.fit_transform(data.motto)</span></span>
>>></span> tfidf_motto</span> =</span> tfidf_vectorizer.fit_transform(data.motto)</span></span>
>>></span> (binary_motto_train,</span></span>
     binary_motto_test,</span></span>
     affil_train,</span></span>
     affil_test)</span> =</span> train_test_split(</span></span>
        binary_motto, data.affiliation,</span> test_size</span>=</span>0.2</span>,</span> random_state</span>=</span>1729</span>)</span></span>
>>></span> (tfidf_motto_train,</span></span>
     tfidf_motto_test,</span></span>
     affil_train,</span></span>
     affil_test)</span> =</span> train_test_split(</span></span>
     tfidf_motto, data.affiliation,</span> test_size</span>=</span>0.2</span>,</span> random_state</span>=</span>1729</span>)</span></span></code></pre>
Now to build the two models we wish to compare.</p>
>>></span> binary_nb.fit(binary_motto_train, affil_train)</span></span>
>>></span> gaussian_nb.fit(tfidf_motto_train.toarray(), affil_train)</span></span></code></pre>Comparing Models</h1>
If this were a real life task, we may not have test data.
To compare our models, then, we won’t resort to the test data just yet;
instead, we’ll use leave one out
cross-validation</a>.</p>
>>> from</span> sklearn.cross_validation</span> import</span> cross_val_score, LeaveOneOut</span></span>
>>></span> binary_cv_score</span> =</span> cross_val_score(binary_nb,</span></span>
                                      binary_motto_train,</span></span>
                                      affil_train,</span></span>
                                      cv</span>=</span>LeaveOneOut(</span>len</span>(affil_train)))</span></span>
>>></span> gaussian_cv_score</span> =</span> cross_val_score(gaussian_nb,</span></span>
                                        tfidf_motto_train.toarray(),</span></span>
                                        affil_train,</span></span>
                                        cv</span>=</span>LeaveOneOut(</span>len</span>(affil_train)))</span></span>
>>></span> print</span>(</span>"Binary CV Score: </span>{0</span>:.5f</span>}</span>"</span>.format(binary_cv_score.mean()))</span></span>
>>></span> print</span>(</span>"Gaussian CV Score: </span>{0</span>:.5f</span>}</span>"</span>.format(gaussian_cv_score.mean()))</span></span>
Binary</span> CV</span> Score:</span>        0.65341</span></span>
Gaussian</span> CV</span> Score:</span>      0.58523</span></span></code></pre>
Though neither of these two classifiers look great, it looks like the binary
one wins on our training data.
Let’s pretend we’ve made the decision to field that classifier in production
and check out its performance on the test data.</p>
Binary Classifier Results</h1>
>>> from</span> sklearn</span> import</span> metrics</span></span>
>>></span> binary_prediction</span> =</span> binary_nb.predict(binary_motto_test)</span></span></code></pre>
Here’s the simplest way of looking at our Binary NB’s performance: a confusion
matrix.
As we can see from this, it looks like our classifier was pretty good at
getting Autobots right, but Decepticons were harder to accurately predict—I
guess this makes sense, since Decepticons are rather sneaky.</p>
>>></span> print</span>(metrics.confusion_matrix(affil_test, binary_prediction))</span></span>
                  Predicted Decepticon    Predicted Autobot</span></span>
 Was Decepticon</span>                      8                   28</span></span>
 Was Autobot</span>                         6                   46</span></span></code></pre>
For a more in-depth measure of the classifier’s performance, we can turn to
sklearn</code>’s classification_report</code>:</p>
>>></span> print</span>(metrics.classification_report(</span></span>
            affil_test,</span></span>
            binary_prediction,</span></span>
            target_names</span>=</span>'Decepticon Autobot'</span>.split()))</span></span>
</span>
             precision    recall  f1</span>-</span>score   support</span></span>
 Decepticon</span>       0.57      0.22      0.32        36</span></span>
    Autobot</span>       0.62      0.88      0.73        52</span></span>
avg</span> /</span> total</span>       0.60      0.61      0.56        88</span></span></code></pre>
According to this, we can see that our classifier is best at identifying actual
Autobots (see the Autobot recall score), but isn’t that great at much else.
For more on how to interpret this table, see Scikit-Learn’s
documentation</a>.</p>
Conclusion</h1>
It looks like we didn’t do that well at classifying Transformer affiliation
from motto; our best classifier only had an F1 score slightly above 50%.
One possibility is that we didn’t have enough data; perhaps a classifier
trained on thousands of samples would do better than ours that had less than
400.
Another possibility is that mottos aren’t that indicative of affiliation to
begin with, and we should look for our predictive power elsewhere.
In any case, it’s kind of fitting to use Machine Learning on Transformers.</p>


Finite Fields: Surprisingly Tricky to Implement Well
2014-09-09T00:00:00+00:00
Apologies</h1>
Sorry for taking so long to post again, fearless reader; I’ve recently moved to
a new house, so most of my outside-of-work-energy has been spent dealing with
all that entails.
This also means that this post is probably not as polished as I may have hoped.</p>
Intro</h1>
While attempting to port some academic code to a, shall we say, cleaner setup
for work, I had some trouble with implementing a piece of somewhat low-level
code that works with finite fields.
Finite fields are important, and the algorithms involved aren’t too terribly
difficult to understand, as these things go.
That being said, I find that they’re surprisingly tricky to implement well.</p>
Finite Fields are Important</h1>
Finite fields are interesting pieces of math in their own right, and involve
interesting topics and ideas like Euclid’s Algorithm and Galois Theory.
They’re quite important for other work, too, since they tend to form the
foundation upon which other ideas and structures build; akin to, say, how high
school geometry talks about points made of pairs of real numbers.</p>
Finite fields have quite important real-world applications, too.
For one example, they form the underlying primitives upon which much of modern
cryptography is based; bitcoin wouldn’t be possible without finite fields,
since it uses
ECDSA</a>.</p>
With that said, they’re hard to implement</h1>
Finite fields are slightly tricky to implement, though, because they are
basically parameterized by some information (the prime characteristic in the
prime field case, and the polynomial modulus in the general case).
Prior to doing any computation within a given field, you have to know exactly
which field you’re dealing with.</p>
Suppose you wanted to write some software to compute in a prime field; what
approach would you take to storing/initializing the characteristic?
Here are some existing options:</p>

NTL’s</a> approach: global initialization of
finite field element type, ZZ_p::init(p)</code>; this is what I used for the
code</a> portion of my
dissertation</a>.
FLINT’s</a> f_q</code> stuff looks similar, though I
haven’t used it.</li>
The Haskell For
Maths</a>
approach: use newtype</code>s for some small examples (F2, F3, F25</code>, etc.),
including extension fields.</li>
The Finite</a>
Fields</a> Haskell
package: type level numbers to hold the characteristic</li>
The Sage</a> approach: on the surface, there is a
class within which you do the work, like GF(17)(3 * 5)</code>.
Behind the scenes, however, this is implemented by (depending on the version
of Sage and the finite field in question) a package like NTL or FLINT (both
mentioned above)</li>
The elegant with-modulus</code> Scheme macro from this Programming
Praxis</a>
exercise.</li>
</ol>
All of these approaches have their drawbacks, however:</p>

NTL requires initializing the field globally before</em> any usage.
I had to chase down crashes early on in writing code for my thesis; it
wasn’t fun</li>
This package only defines a few fields, not a general parameterized way of
easily working over any</em> prime field</li>
This package is quite nice, but only works for prime fields; I have no idea
how to work with, e.g., \(\mathbb{F}_{2^8}\) for
AES</a>.</li>
This is nice on the surface, but that nice interface hides a lot</em> of
different ways of doing things.
It also requires the field object to know about the element object and the
element object to simultaneously know about its parent field object…which
can lead to development headaches.</li>
This is a great way to go, since it clearly marks what field you’re working
with and delimits the scope of those computations nicely—but how many
people know how to effectively craft macros?</li>
</ol>
And all of these problems are only regarding working in a single field!
What about a tower of extensions, or working with other isomorphisms between
fields?</p>
Moral</h1>
Math is full of useful stuff, but it may not be readily apparent how best to
implement those things in a programming language.
In the finite field case, the issue (for me) boils down to effectively dealing
with the extra information that parameterizes our field.
And a lot of potentially useful mathematical objects similarly require knowing
some parameterizing information.
For instance, to work with, say, points on binary Edwards
curves</a>, there are several layers of this going
on: points are parameterized by their coordinates and the curve on which they
lie.
Curves and coordinates, in turn, are embedded in some field; in this case, an
extension field of \(\mathbb{F}_2\).
That extension field, as mentioned above, is parameterized by its
characteristic (2) and its modulus (some polynomial in \(\mathbb{F}_2[x]\)).
My dissertation code is one way of dealing with all of that detail, but I can’t
help but wonder if there is a better way of going about it.</p>


It's Lisp Time
2014-07-07T00:00:00+00:00
Porting academic code is fun, I swear</h1>
I’ve been chewing on a rather tough programming problem at work recently
involving porting code from a collection of different languages to a C++11
production-ish-looking thing.
Really, the project is just to get a cohesive version of the interesting (but
entirely academic, not remotely production-ready) code from a published paper.
Even with the previously mentioned niceties</a>
that C++11 has to offer, it’s still rough going.
And I’ve realized that it’s not just because the code is, ahem</em>, interesting.
It’s that my tools are a bit limiting.</p>
Everything else is too limit</h2>
As it turns out, I’m finding more and more that the things people say singing
the praises of Scheme and Common Lisp might be on to something.
Take, for instance, Peter Seibel’s excellent book Practical Common
Lisp</a>.
In Chapter 9, he puts together a very nice unit testing framework; it’s fully
commented and documented, has great reporting that tells you exactly what went
wrong, and offers a nice little domain specific language for writing tests.
The really awesome part: with comments, indentation, and line breaks, all this
happens in just 26 lines of code</strong>.</p>
Now, I know there are other small unit testing
frameworks</a> out there,1</a></sup> but
this is just amazing.
And it’s just the tip of the iceberg.</p>
All hail the mighty parentheses</h1>
My motivation for wanting more Lisp & Scheme in my coding is twofold, I guess;
first, my work programming, while fun and useful, is constraining in some
senses.
Pretty much any project I write will be either in C, C++, or Python.
If it’s a very data-science-y project, I might get to2</a></sup> use R.
Those languages are great, and they all have their uses, but there’s other
stuff out there, some of it wildly more
powerful</a>.
So the first reason is because I want to use more interesting and powerful
tools.3</a></sup></p>
The second reason is my own edification.
Scheme and Lisp have a decently long history (as far as programming languages
are concerned), and definitely have a library of great work for me to ingest.
There’s the previously mentioned</a>
SICP</a> for Scheme.
I also hope to spend more time on Programming
Praxis</a>, though my attendance has been rather
poor lately.
For Lisp, I hope to read Paul Graham’s On
Lisp</a> and Peter Norvig’s
PAIP</a>.
If I’m feeling particularly adventurous, I hope to read Doug Hoyte’s Let over
Lambda</a>.
I’ve also been intrigued by Clojure recently,4</a></sup> and have been paging through some
of Fogus and Houser’s Joy of Clojure</a>.</p>
Comments/Questions/etc.?</h1>
Does it look like I’ve missed anything, dear reader(s)?
Please let me know if you have other recommendations, etc. for the road to Lisp
enlightenment.</p>



I’ve actually used minunit</code> before, and it’s great for C code. The point is, however, the framework from Seibel’s book is simultaneously more powerful, more expressive, and easier to write tests in (to strand a preposition). ↩</a></p>
</li>

Be forced to? ↩</a></p>
</li>

And on one work project I was forced (for reasons I’ll not go into, but not of my choosing) to use Perl</em>. Kidding aside, it can be a useful tool for some things, but I assure you that what I was asked to build with it is not</strong> one of those things. ↩</a></p>
</li>

I’ll admit I’m a bit reticent about using the JVM, but Clojure has some nice features, so I’m trying to keep an open mind. I’m particularly interested in Incanter</a> as a possible replacement for R and Python for data work. ↩</a></p>
</li>
</ol>
</section>


Unusual Tools in Data Science
2014-05-28T00:00:00+00:00
Isn’t all</em> science “Data Science?”</h1>
I’m an applied research mathematician in my day job, which on a day-to-day
basis puts me somewhere between mathematician, software developer, machine
learning practitioner, and statistician.
In other words, I’m probably what’s called a Data Scientist</em> nowadays.
See the image below for how some people feel about that term;1</a></sup> I’m a bit
ambivalent about it myself.</p>
It seems to be a bit trendy and not have much of a solid definition.
Some people even think the title should be killed
off</a>
entirely.
That being said,</p>

I’m mostly OK with applying a trendy label to myself</li>
I don’t have a better term for the “person who uses software to do
interesting things with data” right now</li>
</ol>
so I’ll stick with it for the time being.</p>
</p>
A series idea</h1>
In my time working in “data science,” I’ve noticed that there are a number of
interesting tools out there that tend to go unnoticed.
Perhaps it’s because people that fit under the data science umbrella have wide
and varied backgrounds, but there seems to be a lack of agreement of best
practices—or a lack of following best practices even if folks agree.
Moreover, there doesn’t seem to be much awareness about the wide array of new,
interesting, or just plain different (computational) tools that are out
there.2</a></sup></p>
There are a number of tools out there that can solve certain problems much more
cleanly, elegantly, or efficiently than the ones</a>
people
usually</a>
reach for.
Though of course the usual tools have a lot of merit—this
book</a> is particularly
nice—there’s definitely room for more tools in the data science tool belt.</p>
Post the first</h1>
The first tool I’ll mention is Haskell for parsing various types of logs and
files.
Several times I’ve ended up having to parse a log or other type of file that,
while it follows a specific format, isn’t one of the usual suspects like SQL or
CSV.
Rather than reaching for regexes,3</a></sup> I contend that this is exactly the type
of problem that Haskell—with
Parsec</a> or (even better)
Attoparsec</a>—can easily
handle.</p>
I won’t make this post into an introduction to parsing with
Attoparsec</a>; there are
already
great</a>
intros out there that do a better job than I could.
I’ll just echo the
experiences</a>
of others out there and let you know that parsing ad hoc (or just new) file
formats with Attoparsec leads to solutions that are</p>

easier to read than regexes</li>
easier to reason about than regexes</li>
easier to come back to and work on/extend later than regexes</li>
… and typically very</em> fast</li>
</ul>
Moral of the story: if you’re doing data science, don’t be afraid to use
different tools than you’re used to; they just might lead to better solutions.</p>



The post from which I stole</s> borrowed that picture is also a pretty good read.</a> ↩</a></p>
</li>

Or perhaps that’s just at my workplace. ↩</a></p>
</li>

Which I’ve also done because Haskell wasn’t an option (the people I was working with only</strong> liked Perl</em>), and it lead to some horrific circumstances similar to this</a>. ↩</a></p>
</li>
</ol>
</section>


Group Theory AND Statistics
2014-02-27T00:00:00+00:00
Diaconis does something cool, video at 11</h1>
Persi</a>
Diaconis</a> is an interesting
character. He dropped out of high school at 14 to become a professional
magician, got a Ph.D. from Harvard, won a MacArthur “genius grant,” and is now
a mathematician and statistician at Stanford University.</p>
He also does interesting math; today I’ll talk about an example from
a paper1</a></sup> of his that I found particularly clever.</p>
Counting voter rankings</h1>
Suppose we are holding an election between five candidates, and we ask voters
to rank all five in order of preference. We’d like to take those rankings of
the five candidates and turn it into a table where the (i, j)</em> entry is the
number of voters who ranked candidate i</em> in place j</em>.</p>
Note that a ranking of five candidates can be thought of as a permutation of
length five, an element of the group \( S_5 \). That is, the permutation
\((5 4 3 2 1)\) corresponds to ranking candidate 5 first, candidate 4 second,
etc. Next, we use the usual linear representation of \( S_5 \); for any given
permutation \( \pi \), let \( \rho(\pi) \) be the 5 x 5 matrix with (i,
j)</em> entry equal to one if \( \pi(i) = j \), zero otherwise.</p>
Define a function \( f: S_5 \rightarrow \mathbb{N} \) such that
\( f(\pi) \) is the number times the vote \( \pi \) was cast. Hence, if 10
people all voted \((5 4 3 2 1)\), then \( f( (5 4 3 2 1) ) = 10 \). Now,
take the Fourier Transform
\[
\hat{f} = \sum_{\pi} f(\pi) \rho(\pi)
\]
Note that \( \hat{f} \) adds up the matrix for each vote times the number
of times that vote appeared; this means that \( \hat{f} \) gives us
a “table” (really a matrix) where the (i, j)</em> entry is the number of times
voters ranked candidate i</em> in place j</em>. This is exactly what we want!</p>
Pithy Python program performs promptly</h1>
Here’s some Python</code> code that computes this.</p>
First off, some useful imports.</p>
from</span> collections</span> import</span> Counter</span></span>
from</span> itertools</span> import</span> permutations</span></span>
import</span> numpy</span> as</span> np</span></span></code></pre>
Next, we define our representation function \( \rho \).</p>
def</span> rho</span>(p):</span></span>
    return</span> np.matrix([np.eye(</span>len</span>(p),</span> dtype</span>=</span>int</span>)[:, i]</span> for</span> i</span> in</span> p])</span></span></code></pre>
After that, some (wicked short!) code to compute the transform
\( \hat{f} \).</p>
def</span> fft</span>(ps):</span></span>
    c</span> =</span> Counter(</span>tuple</span>(p)</span> for</span> p</span> in</span> ps)</span></span>
    return</span> sum</span>(rho(p)</span> *</span> c[p]</span> for</span> p</span> in</span> c)</span></span></code></pre>
The following code also works, but it doesn’t technically follow the definition
above since it hides \( f \).</p>
def</span> fft2</span>(ps):</span></span>
    return</span> sum</span>(</span>map</span>(rho, ps))</span></span></code></pre>
A quick function to normalize a table of counts to percentages instead:</p>
def</span> table</span>(xss):</span></span>
    return</span> 100</span> *</span> xss</span> /</span> np.sum(xss,</span> axis</span>=</span>1</span>)</span></span></code></pre>
And a quick check to see it in action:</p>
if</span> __name__</span> ==</span> '__main__'</span>:</span></span>
    N</span> =</span> 5</span></span>
    S_N</span> =</span> np.array(</span>list</span>(permutations(</span>range</span>(N))))</span></span>
    ps</span> =</span> S_N</span>[np.random.randint(np.factorial(N),</span> size</span>=</span>500</span>)]</span></span>
    fhat</span> =</span> fft(ps)</span></span>
    print</span>(</span>"Permutations:</span>\n{0}\n\n</span>FFT:</span>\n{1}\n\n</span>Percentages:</span>\n{2}</span>"</span>.format(</span></span>
            ps, fhat, table(fhat)))</span></span></code></pre>Fin.</h1>
I’m very excited about this; using group theory to better understand statistics
and machine learning is exactly what I’ve been hoping to do since I finished
grad school.</p>



“Applications of Group Representations to Statistical Problems,” 1990; available here.</a> ↩</a></p>
</li>
</ol>
</section>


N Queens Problem
2013-11-20T00:00:00+00:00
Coding for Interviews</h1>
In order to up my programming chops, I’ve signed up for
this newsletter</a>. It’s been fun,
though I don’t typically have the time/energy to answer the questions.
Somewhat serendipitously, though, they sent out a question that I’ve
poked my nose into recently: the
N Queens Puzzle</a>.</p>
My Solution</h1>
I’ve taken the approach this newsletter seems to prefer: writing my
thinking and solution out by hand. See my write up
here</a>.</p>
Critiquing Your Work</h1>
As I note in my write up, this isn’t the best solution possible. In
fact, my approach is asymptotically just as bad as the naive approach,
basically, by
Stirling’s Approximation</a>.
I was going for shorter code that relied more on the Python standard
library, though.</p>


HyperLogLog is Wicked Cool
2013-10-16T00:00:00+00:00
How to count without counting</h1>
I’ve been messing about with a few different things lately, one of which is the
HyperLoglog
algorithm</a>.
Others have
already</a>
written</a>
about</a>
this</a>
extensively, so I have nothing terribly new to contribute; I just wanted
briefly discuss what it is and show some code.</p>
In Brief</h1>
While it’s a little zen</a>
sounding, HyperLogLog is a way of counting the number of unique items—say,
words in a file—without actually counting them.
That is, it gives a (rather good) estimate of the cardinality of the set of
elements in a given multiset while taking up much less memory than
instantiating the whole set (or a hashed version thereof), and it requires only
a single pass through the data with some smaller post-processing on the end for
a runtime of \( O(N) \).</p>
The larger picture</h1>
I think HyperLogLog is
wicked</a>
cool.
More generally, though, I think it’s a great example of a type of algorithm
that deserves more exploration, especially in this era of big
data</a>: algorithms that trade
resource efficiency for approximate answers.
A big part of what’s so cool about HyperLogLog is that the authors of
this</a> paper give a
highly mathematical and probabilistic description of the possible error and
then mitigate that in the final version of the algorithm.
It’s very impressive.</p>
Code</h1>
For a (slapdash?) C++11 version of the final algorithm, head
here</a>.</p>


SICP is So Good
2013-10-04T00:00:00+00:00
All shutdown and no work makes Graham a dull boy</h1>
I’ve been reading SICP</a> recently in my quest
to extend my Computer Science knowledge.
This is a task I’ve attempted before but never really had the time to devote a
solid chunk of brain-power to.
However, due to recent
events</a>, I find myself with more time on my hands than usual.1</a></sup></p>
SICP to the rescue</h1>
This post will be short, but I imagine I’ll write another sometime soon as I
work through this amazing book.
Of course I’m not the only</a>
one</a>
who</a> thinks so, but SICP is a
good book.
I mean, it’s a really</em> good book.
Within the first chapter, you’ve already used the Fibonacci numbers,
primality testing, Newton’s Method, fixed-points and average damping, and
continued fractions to explore the benefits of having functions as first class
citizens</a>.
This book is amazing.
I suppose that part of what makes it so good is its use of
Scheme</a> as its programming language; this leads to a
minimal barrier between expressing the ideas and executing them.
Even with that said, I’m still in awe of how good this book is.</p>
Case In Point</h1>
As a case in point, I’ll include a quick write up</a> I
did to share with my Dad and his high school students.2</a></sup>
It was inspired by SICP’s handling of an interesting continued fraction.</p>



It’s not all SICP all the time for me; I have to tackle a truly impressive list of chores as well. ↩</a></p>
</li>

I really need to get back to work. ↩</a></p>
</li>
</ol>
</section>


Richard Stanley, Catalania, and Haskell
2013-09-15T00:00:00+00:00
Richard Stanley, Captain of Counting</h1>
I recently received an email about a math
conference</a> in honor of Richard
Stanley’s 70th birthday.
For those of you who don’t know, Stanley is an expert in combinatorics and
author of the two volume magnum opus Enumerative
Combinatorics</a> which (among a bunch of
other very interesting math) champions the idea of the combinatorial
proof</a>—a way of proving a
combinatorial identity by showing that both sides of the equation are in
different fact ways of counting the same thing.</p>
Stanley’s Enumerative Combinatorics also goes into great detail about the
Catalan Numbers</a>, a sequence
of numbers that arises in tree enumeration problems.
In fact, Stanley’s interest in these numbers—which he dubs ’‘Catalan
disease’’ or ’‘Catalania’’—goes so far that he offers 66 combinatorial
interpretations of them in EC vol. 2 and dozens more in an addendum published
online</a>.
See the OEIS
page</a>
for more.</p>
Catalania, meet Haskell</h1>
In the interest of learning more Haskell, I decided to take some inspiration
and play with Catalan numbers in that decidedly mathematical programming
language.
To start off with, I have some helpful imports and auxiliary functions:</p>
import</span> Test.QuickCheck</span> (</span>quickCheck</span>)</span></span>
</span>
-- helpers</span></span>
binom</span> :: Integer -> Integer -> Integer</span></span>
binom n k </span>=</span> product [k </span>+</span> 1</span>..</span>n]</span> `div`</span> product [</span>1</span>..</span>n </span>-</span> k]</span></span>
</span>
factorial</span> :: Integer -> Integer</span></span>
factorial </span>=</span> product </span>.</span> enumFromTo </span>2</span></span>
</span>
pairs</span> ::</span> [a]</span> -></span> [(a, a)]</span></span>
pairs xs </span>=</span> [(x, y)</span> |</span> x </span><-</span> xs, y </span><-</span> xs]</span></span>
</span></code></pre>
The QuickCheck</a> import will be
important later.</p>
Next, with the help of Rosetta
Code</a> and Wikipedia, I
have the following Catalan Number definitions:</p>
-- Catalan definitions</span></span>
cat0</span> ::</span> [</span>Integer</span>]</span></span>
cat0 </span>=</span> map (</span>\</span>n </span>-></span> product [n </span>+</span> 2</span>..</span>2</span> *</span> n]</span> `div`</span> product [</span>2</span>..</span>n]) [</span>0</span>..</span>]</span></span>
</span>
cat1</span> ::</span> [</span>Integer</span>]</span></span>
cat1 </span>=</span> 1</span> :</span> map c [</span>1</span>..</span>]</span></span>
  where</span> c n </span>=</span> sum </span>$</span> zipWith (</span>*</span>) (reverse (take n cat1)) cat1</span></span>
</span>
cat2</span> ::</span> [</span>Integer</span>]</span></span>
cat2 </span>=</span> scanl (</span>\</span>c n </span>-></span> c </span>*</span> 2</span> *</span> (</span>2</span> *</span> n </span>-</span> 1</span>)</span> `div`</span> (n </span>+</span> 1</span>))</span> 1</span> [</span>1</span>..</span>]</span></span>
</span>
cat3</span> ::</span> [</span>Integer</span>]</span></span>
cat3 </span>=</span> map c [</span>0</span>..</span>]</span> where</span></span>
  c n </span>|</span> n </span><=</span> 0</span>    =</span> 1</span></span>
      |</span> otherwise </span>=</span> binom (</span>2</span> *</span> n) n </span>-</span> binom (</span>2</span> *</span> n) (n </span>+</span> 1</span>)</span></span>
</span>
cat4</span> ::</span> [</span>Integer</span>]</span></span>
cat4 </span>=</span> map c [</span>0</span>..</span>]</span></span>
  where</span> c n </span>=</span> sum (map (</span>\</span>k </span>-></span> binom n k </span>^</span>2</span>) [</span>0</span>..</span>n])</span> `div`</span> (n </span>+</span> 1</span>)</span></span>
</span>
cat5</span> ::</span> [</span>Integer</span>]</span></span>
cat5 </span>=</span> map (</span>\</span>n </span>-></span> binom (</span>2</span> *</span> n) n </span>`div`</span> (n </span>+</span> 1</span>)) [</span>0</span>..</span>]</span></span>
</span>
cat6</span> ::</span> [</span>Integer</span>]</span></span>
cat6 </span>=</span> map c [</span>0</span>..</span>]</span></span>
  where</span> c n </span>=</span> factorial (</span>2</span> *</span> n)</span> `div`</span> (factorial (n </span>+</span> 1</span>)</span> *</span> factorial n)</span></span></code></pre>
Finally, we’d like to show (or at least, verify for a certain number of test
cases) that these varying definitions all describe the same thing; to do that,
I decided to use a bit of QuickCheck’s magic to test various items from these
lists for equality:</p>
-- Check that these look the same</span></span>
prop_equal</span> :: Int -> Bool</span></span>
prop_equal n </span>=</span> all p </span>$</span> pairs cats</span></span>
  where</span></span>
    cats </span>=</span> [cat0, cat1, cat2, cat3, cat4, cat5, cat6]</span></span>
    p (xs, ys)</span> =</span> xs </span>!!</span> n' </span>==</span> ys </span>!!</span> n'</span></span>
    n' </span>=</span> n </span>`mod`</span> 1000</span></span>
</span>
main</span> :: IO</span> ()</span></span>
main </span>=</span> quickCheck prop_equal</span></span></code></pre>
This isn’t perhaps the most elegant check—note that I restrict the integers
tested to \( \lbrace 0, 1, \ldots, 999 \rbrace
= \phantom{ }^\mathbb{Z} /_{1000 \mathbb{Z}} \) to make sure
that indices are nonnegative and we don’t overtax my poor laptop’s CPU.
Of course, I’m open to criticism from any Haskell/QuickCheck gurus who’d like
to help me tighten up my code.
Feel free to download the
code</a>.</p>
To Be Continued</h1>
That was fun; programming in Haskell almost always is, for me, and I’m by no
means an expert.
However, it leaves a little to be desired.
The program above at best demonstrates that the various infinite lists agree at
several (pseudo-)randomly chosen indices.
I suppose that taking this to the next level would involve some sort of
automatic theorem proving
assistant—Agda</a> comes to
mind, due to its relationship with Haskell.
That being said, using something like Agda will at best give me algebraic</em>
proofs that \( cat_i = cat_j \) by exploiting various mathematical
properties of the definitions of two lists.
I highly doubt that it would be able to produce a combinatorial</em> proof that
two descriptions of the Catalan numbers are the same, though I’d be very
happy to be proven wrong.</p>


Dear Statistics, Let's be Friends
2013-08-27T00:00:00+00:00
Dear Statistics,</p>
I apologize for not having taken you seriously until this year.
I said that you aren’t Mathematics; that’s true, but not in the (derisive) way
I meant it.
You aren’t Math in the same way that Physics isn’t Math—you sure use a whole
lot of Math to do some very interesting and important things.
Really, you are the science of data;1</a></sup> you enable us to learn and make
decisions in a noisy, uncertain world.</p>
I changed my mind when my eyes were opened to all the useful things you can do,
of which there is a
whole</a>
enormous,</a>
gigantic,</a>
humongous,</a>
impressive</a>
list.2</a></sup>
In my ignorance and arrogance, I had no idea.
Fortunately, I had the honor of taking classes with some
incredible</a>
professors in graduate school that opened my eyes to what your buddy
Probability had in store; once I stopped hating on Probability, it was only a
matter of time until I gave you a second look.</p>
By and large, though, one of the biggest things that helped me appreciate you
better is my developing programming abilities.
Rather than working by hand, I can now use things like
R</a> to, say, flip a billion unbiased virtual coins
in the blink of an eye.
Let’s be honest, you involve a lot of computation; what better tool could I use
than a computer?</p>
That last thought reminds me: arguably one of the reasons I never took you
seriously is the poor way in which you’re taught in schools.
I think you are subject to the problems mentioned in Lockhart’s
Lament</a> even
more than Math is.
A lot of teachers seem to think you consist solely of blinding applying
formulas without understanding; as a Mathematician who loves learning and
understanding things, I too lament this deplorable idea.
As Richard Hamming</a> once said,
the purpose of computing is insight, not numbers.
I now know it makes sense to use squared distance from the mean for variance
instead of absolute distance, because doing calculus with absolute value is a
pain.
I now know that your best friend Probability is actually
Analysis</a> in disguise.
Of course, there isn’t a need to know graduate level Analysis to successfully
apply you, but I sure feel better knowing why I’m doing what I’m doing.
There has to be a happy medium somewhere closer to developing understanding
than what passes for Math & Stats education in most3</a></sup> schools nowadays.</p>
To sum up, Statistics: I apologize. Let’s be friends.</p>
Best,</p>
Graham</p>



I’m avoiding calling you “Data Science,” since that’s too trendy, but you really are. ↩</a></p>
</li>

I claim that if Machine Learning isn’t a proper sub-discipline of Statistics, it at least uses so much statistical knowledge that it might as well be counted as a win. ↩</a></p>
</li>

Most American schools, perhaps? My perspective is limited to my own country. ↩</a></p>
</li>
</ol>
</section>


Python to C++; a silly idea?
2013-08-12T00:00:00+00:00
EDIT</h1>
In the more than ten years since I first wrote this,</p>

I have learned a lot about programming, programming languages, compilers, interpreters, and more; and</li>
The ecosystem around this kind of idea has changed a lot! See, for instance, mojo</code></a>.</li>
</ol>
I’ll leave the rest of this post untouched; please forgive past Graham’s naivete!</p>
Python type annotations</h1>
In the 3.x series of the Python</a> language, there’s a
powerful,
unused</a>
feature in the standard: type
annotations</a>.
As they currently stand, type annotations don’t seem to do anything except set
some attributes of the function to which they’re applied; however, I’ve been
thinking of a possible use for them: compiling Python to C++.</p>
I’m aware of the amazing Cython</a> project, and
various</a> other</a>
options</a> when it comes to compiling code
for faster Python runtimes.
There’s even Shed Skin</a>, a compiler that
takes much of the 2.x Python standard to C++ already.
Thus, this idea is nothing more than an excuse for me to learn more computer
science, specifically compilers (and lexers and parsers, oh my!).
However, besides Python 3.x’s type annotations, there’s one other feature none
of these options use:1</a></sup>
C++11</a>, the new C++ standard.</p>
C++11</h1>
C++11 is quite interesting; Bjarne Stroustrup, the creator of C++, says</p>

Surprisingly, C++11 feels like a new language: The pieces just fit together
better than they used to and I find a higher-level style of programming more
natural than before and as efficient as ever.</p>
</blockquote>
There’s a huge amount of things that have changed in the new standard, but some
of my favorites are type
inference</a>, lambda
functions</a>,
and ranged-based for
loops</a>.  These new
features make C++ a much more pleasant language to work with, in my opinion; in
fact, I like it so much that I used it for the code
portion</a> of my dissertation.</p>
The above-mentioned type inference, coupled with alternative function
syntax</a>,
are what gave me the (probably silly) idea of trying to compile Python 3.x code
to C++11 code.</p>
Suggestive examples</h1>
Here are some slightly suggestive code examples that got me thinking.
The idea would be to somehow compile (perfectly valid) Python 3 code like this:</p>
def</span> f</span>(x:</span>int</span>, y:</span>int</span>) -></span> int</span>:</span></span>
    return</span> 2</span> *</span> x</span> +</span> y</span></span></code></pre>
into equivalent2</a></sup> C++11 code:</p>
auto</span> f</span>(</span>int</span> x</span>,</span> int</span> y</span>) -></span> int</span> {</span></span>
    return</span> 2</span> *</span> x </span>+</span> y;</span></span>
}</span></span></code></pre>
Here’s a slightly more involved example: what if we could compile a (very</em>
tiny) Python class</p>
class</span> R2Point</span>(</span>object</span>):</span></span>
    __slots__</span> =</span> (</span>'x'</span>,</span> 'y'</span>)</span></span>
</span>
    def</span> __init__</span>(self, x:</span>int</span>, y:</span>int</span>) -></span> None</span>:</span></span>
        self</span>.x</span> =</span> x</span></span>
        self</span>.y</span> =</span> y</span></span>
</span>
    def</span> __add__</span>(self, other:R2Point) -> R2Point:</span></span>
        return</span> R2Point(</span>self</span>.x</span> +</span> other.x,</span> self</span>.y</span> +</span> other.y)</span></span></code></pre>
into a C++11 struct</p>
struct</span> R2Point</span> {</span></span>
    int</span> x, y;</span></span>
    auto operator</span>+</span>(</span>R2Point</span> other</span>) -> R2Point {</span></span>
        return</span> R2Point{x </span>+</span> other.x, y </span>+</span> other.y};</span></span>
    }</span></span>
};</span></span></code></pre>
We can even handle Python lambdas3</a></sup>, to some extent.
This,</p>
f</span> = lambda</span> x:</span> 2</span> *</span> x</span></span>
xs</span> =</span> xrange</span>(</span>10</span>)</span></span>
ys</span> =</span> []</span></span>
for</span> x</span> in</span> xs:</span></span>
    ys.append(f(x))</span></span>
print</span>(</span>sum</span>(ys))</span></span></code></pre>
might become</p>
template</span> <</span>typename</span> T</span>></span> auto</span> f </span>=</span> [](</span>const</span> T</span>&</span> x</span>) {</span> return</span> 2</span> *</span> x; }</span></span>
auto</span> xs </span>=</span> std</span>::</span>vector</span><</span>int</span>>(</span>10</span>);</span> std</span>::</span>iota</span>(xs.begin, xs.end,</span> 0</span>);</span></span>
auto</span> ys </span>=</span> std</span>::vector</span><decltype</span>(xs.</span>front</span>())</span>></span>;</span></span>
for</span> (</span>auto</span> x: xs) {</span></span>
    ys.</span>push_back</span>(</span>f</span>(x));</span></span>
}</span></span>
std</span>::cout </span><<</span> std</span>::</span>accumulate</span>(ys.</span>begin</span>(), ys.</span>end</span>(),</span> 0</span>)</span> <<</span> std</span>::endl;</span></span></code></pre>
Admittedly, these aren’t very involved examples; the code is quite short, and
they don’t involve translating Python idioms like list comprehensions into C++.
I still find them suggestive, though (at least, the first two).</p>
Something else</h1>
Like I said, these aren’t terribly involved examples and don’t demonstrate
changing idiomatic Python code into idiomatic C++11 code, really.
If the Python to C++ idea doesn’t pan out, though, there’s always the option of
something analogous to the relationship between
CoffeeScript</a> and
JavaScript</a>.
Since one of the common
criticisms</a> of
C++ is that it is too complex, a simpler front end that compiles to a C++
back end might be interesting.
That being said, I think that starting fresh with idiomatic C++11 might be
enough of a simplification in some cases.</p>
In the long run</h1>
In the long run, I doubt this idea is really going to take off; Pythonistas
like Python for their reasons, and C++ coders like their language for their
own.
More than anything, any potential transpiler of Python to C++ I develop would
be a chance for me to learn more about how programming languages are
implemented and how compilation works.
I had a lot of fun reading this</a> stellar book,
and might take its approach to this problem.
I thought it was interesting how similar the syntax of the two languages
can look in their latest versions, and that got me wondering.
If I take this idea anywhere, I’ll be sure to write about it.</p>



As far as my cursory research has shown; please feel free to correct me if I’m wrong! ↩</a></p>
</li>

Well, largely equivalent; Python has arbitrarily large integers, which C++ can’t match natively (even using intmax_t</code>). ↩</a></p>
</li>

Which are weak</a>—some might say broken</a>—compared to C++11’s (or Lisp’s, Scheme’s, Haskell’s, etc.). ↩</a></p>
</li>
</ol>
</section>


Find X[i] = i in an array (Programming Praxis)
2013-08-06T00:00:00+00:00
Introduction</h1>
The inimitable Programming Praxis</a>, to which I
owe much of my programming abilities1</a></sup>,
posted</a> an
exercise that was seemingly simple but has some surprising depth to it.
Since it’s been a little while since the problem was posted, I have few qualms
about discussing it here.
However, don’t read on if you prefer to solve the problem yourself first.</p>
The question</h1>
Programming Praxis asks,</p>

Given an array X[0 .. n - 1]</code> of integers sorted into ascending order with
no duplicates, find an array item that is also its index, so that X[i] = i</code>.</p>
For example, X[3] = 3</code> in the array shown below:</p>
i     0 1 2 3 4 5</span></span>
X[i] -3 0 1 3 5 7</span></span></code></pre></blockquote>
Official solutions</h1>
The first official solution is a simple linear search through the array; start
at index 0</code> and walk up until you either find an i</code> such that X[i] = i</code> or
hit the end of the array.
This solution, however, is dismissed in favor of a better one that uses a
binary search.
Since the array was presumed to be sorted, a binary search finds the answer in
logarithmic time instead of linear time.
The author points out that if you only stick with the linear answer in an
interview, chances are you won’t make the cut.</p>
A different approach</h1>
I’ve been thinking about a different interview scenario in which you probably
still won’t make the cut if you stick with the linear solution.
Suppose you’re trying to claim competency in the R statistical programming
language</a>.
If you submit the linear solution in R, you’d still fail (or at least, not
succeed with flying colors).
However, I think that the binary search approach is wrong here as well; much of
R depends on thinking over a whole collection at a time, through
vectorization</a>.
For example, one can quickly ask which elements of the vector V</code> are less than
ten by typing V < 10</code>.</p>
Here’s my R solution in full:2</a></sup></p>
X</span> <-</span> c</span>(</span>-</span>3</span>,</span> 0</span>,</span> 1</span>,</span> 3</span>,</span> 5</span>,</span> 7</span>)</span></span>
X</span>[</span>X</span> ==</span> seq</span>(</span>length</span>(</span>X</span>))</span> -</span> 1</span>]</span></span></code></pre>
The first line simply assigns the example vector to the variable X</code>, as we
might expect; the c()</code> function combines or concatenates its arguments.
Next, let’s talk about the sequence function; seq(n)</code> returns the vector 1 2 3 ... n</code>; this means that seq(length(X))</code> gives the sequence of integers from
1</code> to the length of our vector x</code>, and subtracting one from it yields 0 1 2 ...  length(X) - 1</code>.
That subtraction was the first time we took advantage of R’s implicit loops;
the second time comes when we compare X</code> to this new vector we’ve created.
The expression X == seq(length(X)) - 1</code> yields a boolean vector which is
True</code> wherever X[i] == i</code> and false wherever they differ, just as we wish.
Finally, the outer X[]</code> returns those entries in X</code> indexed by the inner
expression, i.e. only those X</code> values corresponding to a True</code> value.</p>
Though I’m not an R expert, I think this is pretty idiomatic R programming and
would be a more acceptable answer in an interview geared toward demonstrating
knowledge of the statistical programming language—I’m trying to learn,
though, so if anyone has a better solution, please let me know!
What’s more, it will probably end up being faster than the logarithmic binary
search just due to the way R is designed; it’s optimized for these vectorized
expressions.
While explicit looping and conditionals can be done in R, they tend to be much
slower.
If you reach the point where the logarithmic solution in slower R code catches
up with the linear implicit looping one, chances are your problem is big enough
that R isn’t the language to use anyway.</p>
Brevity and elegance</h1>
I also offered a solution in the J programming
language</a> which operates
in the same way but is probably harder to read—harder for the uninitiated, at
least.
Per the Wikipedia page, J is an array-oriented functional programming language
that grew out of Kenneth Iverson’s
APL</a>.
As a mathematician, I find these languages fascinating; they endeavor to be
both “executable math” and better</em> math.
I’ll try to explain both of these points a bit, but I’m sure I won’t do them
justice.</p>
As far as “executable math” is concerned, suppose we wanted, say, the sum of
the first 17 natural numbers.
In plain ol’ C, we’d go with something like this:</p>
int</span> sum </span>=</span> 0</span>;</span></span>
for</span> (</span>int</span> i </span>=</span> 0</span>; i </span><</span> 17</span>;</span> ++</span>i) {</span></span>
    sum </span>+=</span> i;</span></span>
}</span></span>
printf</span>(</span>"</span>%d\n</span>"</span>, sum);</span></span></code></pre>
Expressing the same thing in J is much more declarative.
In J, i.17</code> is the first 17</code> natural numbers.
Next, +/</code> is sum, since it’s plus
folded</a> over its
argument.
Hence, +/i.17</code> is the sum we want; compare that to the brevity of
\[ \sum_{i = 0}^{16} i \]
The J code says what the C loop tries to say, but much more succinctly; there’s
very little in the way of taking the mathematical expression—which in
un-typeset LaTeX reads something like “the sum from i = 0</code> to 16</code> of
i</code>”—and having the computer execute it.</p>
As for the “better math” claim, APL originally grew out of a mathematical
notation that Kenneth Iverson created.
I won’t go into much detail here, but will give one example stolen from this
presentation</a>.
In APL, all evaluation proceeds from right to left; there is no order of
operations, so no need for parentheses to insure that, e.g., an addition
happens before a multiplication.
As such, Horner’s rule for polynomial evaluation of
\( a + bx + cx^2 + dx^3 + ex^4 \)
goes from this:
\[ a + x \cdot(b + x \cdot (c + x \cdot (d + x \cdot e))) \]
to this:
\[ a + x \cdot b + x \cdot c + x \cdot d + x \cdot e \]
In his 1979 Turing Award Lecture titled Notation as a Tool of
Thought</a>, Iverson makes a strong
argument for improving mathematical and programming notation by combining them
to better serve our needs.
Though I may not be entirely sold on this just yet, it certainly is a very
interesting idea and I will definitely explore it more.</p>
A J solution</h1>
Now for my J solution to the original question:</p>
(I.@:=i.&#) _3 0 1 3 5 7</span></span></code></pre>
or, if we assign the array the name X</code>,</p>
X =. _3 0 1 3 5 7</span></span>
I.X=i.#X</span></span></code></pre>
I won’t discuss the first one here (to be honest, I’m very new to J and don’t
understand it well enough).
However, the second one should look an awful lot like the R solution we saw
earlier.
Taking it in pieces, #X</code> returns the length of the vector X</code>.
As we saw above, i.</code> counts from 0</code> up to one less than its argument, so
i.#X</code> yields the vector \(\langle 0, 1, 2, \cdots \# X - 1\rangle\)
Next, X=i.#X</code> returns a vector of ones and zeros indicating where X</code> is equal
to this vector (i.e. where X[i]</code> equals i</code>).
Finally, the operator I.X</code> uses that boolean vector to select values from X</code>,
much as X[]</code> did in the R solution.</p>
Summing up</h1>
I think Programming Praxis is right in saying that in a typical programming
interview, the linear approach to this problem just won’t cut it.
However in a stats-geared interview, I think the logarithmic answer might be
less than stellar, too.
Though my R solution doesn’t make use of the fact that X</code> was supposed to be
sorted (nor does my J solution), it is much more idiomatic and demonstrates a
better understanding of what R is supposed to do and what it does well.
Both of my solutions also have the added advantage of finding all possible
answers in an array, not just one.</p>
Postscript</h1>
In the discussion section at the end of Formalism in
Programming Languages</a>, Iverson
answers a question from Dijkstra that sounds very familiar:</p>

Dijkstra</em>: How would you represent a more complex operation, for example,
the sum of all elements of a matrix M</strong> which are equal to the sum of the
corresponding row and column indices?</p>
Iverson</em>:3</a></sup> \( ++/(\mathbf{M} = \iota^1 {\Box \atop +} \iota^1) //
\mathbf{M} \)</p>
</blockquote>



Such as they are. ↩</a></p>
</li>

I’ve changed all xs</code> in my submitted solution to X</code> to be more in line with the question. ↩</a></p>
</li>

The “box atop plus” symbol above isn’t quite right, but regretfully I don’t know how to make the appropriate APL symbol in the limited LaTeX afforded to me by MathJax & \( \KaTeX \). ↩</a></p>
</li>
</ol>
</section>


Initial Post
2013-07-30T00:00:00+00:00
So… here’s my first post.
I’ve been inspired to keep a digital notebook/record of sorts, mostly to keep
me honest about continuing my education; the theory is that if I have something
to keep me accountable, some place to post my work, then I’ll keep working.</p>
Since this is my first post, I’m going to use this space for two things:</p>

as a play sandbox to see how things render</li>
to dicuss some of the things I hope to write about</li>
</ol>
Sandbox</h1>
My father the math-teacher-extraordinaire says that Euler’s
identity \( e^{i\pi} + 1 = 0 \) is proof of the existence of God.
Here’s some Python</code> code that tries to verify this fact:</p>
from</span> cmath</span> import</span> exp, pi</span></span>
print</span>(exp((</span>0</span> +</span> 1</span>j</span>)</span> *</span> pi)</span> +</span> 1</span>)</span></span></code></pre>
Due precision issues, this probably won’t come out to zero.
However, it sure looks darn close.</p>
Serious Note</h1>
My dissertation focused on applying a specific type of abstract algebra to
cryptography (for the technically inclined: the use of Edwards curves,
specifically binary Edwards curves, in elliptic curve cryptography; if you have
trouble sleeping, I can provide a copy of my dissertation).
As such, I have a soft spot for computational group theory, abstract algebra in
general, and cryptography.
Though I have no problem geeking out on these topics at the drop of a hat, I’m
hoping to broaden my horizons a bit—writing a dissertation is a bit draining.
I’m particularly interested in learning more probability, statistics, and
machine learning; though as of this writing I have a passing understanding of
these topics, I hope to dive into them much more deeply.</p>
Graham Enos

About

Tempest in a Teacup

Parallel Means

APL Pi Day

Adding up uniform variates

Different Implementations of a Tiny Programming Language

Here, Have a Fractal

Tropical Semiring in Dyalog APL

Evaluating quantum generative models via imbalanced data classification benchmarks

A Rusty Quantum Interpreter

Random Art with a Tiny Language

Kompreni: Understanding OCaml with Abstract Algebra Examples

Scum: write me a Scheme-ish

Postscript</h1> 👋 to the former coworker who enjoined me to blog more! I miss working with you, friend.</p>

Algebraic Structure ⇒ Computational Benefits, Now With 100% More Oxidization

Time Traveling Key-Value Store, This Time in Rust

Testing</h1> Rust’s (and cargo</code>’s) approach to testing is a pleasure to use. We can do regular unit-style tests:</p>

Synthetic weather radar using hybrid quantum-classical machine learning

CvRDT Exposition in Rust

Tiny REST API Demo in Python

PostgreSQL Makes Delicious Dog Food

Silly Shenanigans

Introduction</h1> In an effort to both post more than once a year and clean out some silly things from my ~/tmp</code> directory, here’s a quick post about two programming shenanigans. Perhaps unsurprisingly due to its history of jocularity, both originate with Perl.</p>

Time Traveling Key-Value Store, Again

A Time Traveling Key-Value Store

Appendix</h1> Here’s the flake.nix</code> file to specify the W O R L D:</p>

Slides: Algebraic Structure, Computational Benefits

Fourier Transform in Clojure, Reproducibly with Nix

Two Short Python Things

Sorry</h1> Apologies, as ever, for taking so</em> incredibly long between posts. That said if the inimitable Roger Peng can get away with letting some time lapse while working on a project, perhaps you can forgive me?</p>

Cross Compiling Static Rust Binaries for Linux on OS X

Compile & Deploy</h1> To compile your executable for the server, run</p>

ScalaCheck and the Unkillable Trash

New Job, Hire in DC, and a Short Revisit of Finite Fields

Hola</h1> Is it poor form to start each post with an apology for the time elapsed since the previous one? Either way, hello dear reader. Beyond being the father of a rambunctious young’un, things have changed around here since we last spoke.</p>

Learning By Building: A Simple Neural Network Implementation

Small baby, no sleep make Graham a dull blogger</h1> Apologies, readers, for my long absence. My family grew this last year, so I’ve had less time for writing.</p>

Predicting Transformer Affiliation from Motto

Bring out the Python</h1> Let’s start up IPython and start digging.</p>

Scikit-Learn</h1> Let’s import what we’ll need: two vectorizers for transforming text into matrices, and the two naive Bayes classifiers we’ll compare. We’ll also grab the word_tokenize</code> function from NLTK</code>.</p>

Finite Fields: Surprisingly Tricky to Implement Well

Apologies</h1> Sorry for taking so long to post again, fearless reader; I’ve recently moved to a new house, so most of my outside-of-work-energy has been spent dealing with all that entails. This also means that this post is probably not as polished as I may have hoped.</p>

It's Lisp Time

Unusual Tools in Data Science

Group Theory AND Statistics

N Queens Problem

HyperLogLog is Wicked Cool

Introduction</h1>
In an effort to both post more than once a year and clean out some silly things from my `~/tmp</code> directory, here’s a quick post about two programming shenanigans. Perhaps unsurprisingly due to its history of jocularity, both originate with Perl.</p>`

Hola</h1>
Is it poor form to start each post with an apology for the time elapsed since the previous one? Either way, hello dear reader. Beyond being the father of a rambunctious young’un, things have changed around here since we last spoke.</p>

Small baby, no sleep make Graham a dull blogger</h1>
Apologies, readers, for my long absence. My family grew this last year, so I’ve had less time for writing.</p>

Apologies</h1>
Sorry for taking so long to post again, fearless reader; I’ve recently moved to a new house, so most of my outside-of-work-energy has been spent dealing with all that entails. This also means that this post is probably not as polished as I may have hoped.</p>
