Jekyll2023-11-21T21:17:17+00:00https://channingwalton.github.io/feed.xmlChanning WaltonBlogs and other musingsChanning WaltonRumblings in Scala2023-05-02T00:00:00+00:002023-05-02T00:00:00+00:00https://channingwalton.github.io/posts/2023-05-02/scala<p>Here are some thoughts I posted about Scala on <a href="https://types.pl/@channingwalton/110299339793580597">Mastodon</a>.</p>
<p>There is quite a bit of talk about Scala and it’s future it seems.
There are thoughts about academia driving the language for its own noble goals but losing sight of the consequences for industrial size codebases in the wild and the ecosystem as a whole.</p>
<p>It’s a justifiable opinion. For example, the IDE support has been a long struggle and is pretty good for 2, but it has taken a big step backwards for 3. It’s become a tiring and unnecessary impediment to just getting things done for the average programmer who doesn’t want to repeatedly quit the IDE, killall java, or reimport projects because “something weird has happened”. We don’t have time for that crap.</p>
<p>Now you might all say I don’t know what I’m talking about, that with some clever solutions to X and Y and using a proper editor, everything will be great. Maybe, but having worked hard at trying a lot of these tools and IDEs, I’ve found little joy with Scala 3. I’ve also tried migrating systems to Scala 3 several times and gave up because it is a herculean task with considerable risk to the systems.</p>
<p>I’m not a type astronaut or contributor to libraries like the type level ecosystem – I’m not smart enough. But, I think I can say I’m not bad at building systems by leveraging the work of those geniuses. And, I am pretty happy working with Scala 2 and enjoying the improvements around the ecosystem that are happening all the time.</p>
<p>I’m enormously grateful to all those really smart people working very hard, for free, to make it possible for me to have an enjoyable career.</p>
<p>The success of the language depends critically on an ecosystem sustained by very smart people giving up their free time, and so it is imperative that changes to the language do not exhaust their good will and patience. I fear that Scala 3, whilst having many great features, was too much too fast for everyone.</p>Channing WaltonHere are some thoughts I posted about Scala on Mastodon.Hiring a programmer? Make some tea!2023-04-24T00:00:00+00:002023-04-24T00:00:00+00:00https://channingwalton.github.io/posts/2023-04-24/jobs<p>Whilst looking for my next job, I’m finding that the hiring process has become little more than an exercise in matching tags.</p>
<p>A few years ago I responded to a question on Quora, <a href="https://qr.ae/pyRRiu">How do I explain to non-programmers how complex, time-consuming, and error-prone software development is?</a>.</p>
<p>The essence of my response was to describe how an apparently simple process of making tea is not as simple as it first seems, and that the thinking behind it contains thousands of details that programmers
need to consider.</p>
<p>It was a popular response that resonated with a lot of developers.</p>
<p>A few days ago I <a href="https://twitter.com/channingwalton/status/1650133214126645254">tweeted</a> a joke about how we’d hire drivers like we hire programmers. But I am going to keep the tea analogy going and recast it:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>If we advertised for someone to make tea like devs.
Required:
- three years of brewing experience
- can boil a kettle
- knowledge of cold water and electricity, particular mains AC
- experience with Yorkshire tea bags, PG tips. Tetley a bonus
- proven and demonstrateable skills chopping tea leaves
- chemistry of oxidation converting polyphenols into
compounds such as theaflavins and thearubigins
- Bone china cup making
</code></pre></div></div>
<p>For developers, a lot of job adverts read that way.</p>
<p>A common thing that comes up is whether the candidate has previously used a particular product like a database or queue. Those tools are big and complicated, and appear in architecture diagrams and are rightfully critical to the system. So, it is natural to ask if the candidate has used them before.</p>
<p>But here’s the thing, from a programmers perspective <em>the specific products are not a big deal</em>. The difference from one database to the next is trivial and has very little impact on what the programmer is doing day to day.</p>
<p>If those systems have a large impact daily, then you have a much bigger problem, your architecture or use of those products is inappropriate, and a good developer will debug it.</p>
<p>Some products have novel features that others don’t, and any developer will learn those features quickly (hours), and apply them to the problem at hand.</p>
<p>It feels like an easy shortcut to match candidates against products and techniques (TDD, ATDD, BDD, OO, FP, etc.), but by doing so you’re missing out on good people which will cost you more in the long run.</p>
<p>Focus on what really matters. Is the candidate passionate about solving problems, building what users need, striving for simplicity, producing bug free systems, and know how to do those things?</p>
<p>Most importantly, can the team have an enjoyable cup of tea with the candidate whilst discussing how the system and processes you’re building could be improved?</p>Channing WaltonWhilst looking for my next job, I’m finding that the hiring process has become little more than an exercise in matching tags.You want to use AI? Learn to program first!2023-04-19T00:00:00+00:002023-04-19T00:00:00+00:00https://channingwalton.github.io/posts/2023-04-19/ai<p>Whilst using AI tools to help me write software, it struck me that these tools in their current form, are only safe in
the hands of experienced developers.</p>
<p>There are many examples of expert level tools that are dangerous in the hands of novices, such as power tools in
carpentry or advanced surgical instruments. Novices need to earn the right to use those tools and prove they know how to
use them safely.</p>
<p>And so it is with the current crop of AI tools which are dangerous in naive hands because they hallucinate, and produce
answers that need careful handling and scrutiny.</p>
<p>Programming has some natural protections that other industries don’t. For example, the code produced by AI often doesn’t
compile or is just clearly nonsense. But passing the bar exam is a party trick, would you take legal advice from
an AI? If you would I have a bridge to sell you, perhaps you can ask the AI to draft the details.</p>
<p>Code compilation isn’t the big issue, AI often produces plausible looking code that does compile and run. The
problem is that solutions often have subtle bugs, or worse, push designs in the wrong direction leading to expensive
time-wasting and dead ends. Remember, it doesn’t have a big picture … of anything!</p>
<p>It takes experience to see the solutions offered by these tools, find what’s useful, and apply them to a much larger and
more complex context in the design and architecture of the system being built.</p>
<p>My advice if you’re starting out in programming is to learn how to program, design and architect solutions, before
looking for shortcuts that aren’t there.</p>
<p>Embrace the grind, pay the fee, learn the craft.</p>Channing WaltonWhilst using AI tools to help me write software, it struck me that these tools in their current form, are only safe in the hands of experienced developers.Introducing the Ritronome2023-04-04T00:00:00+00:002023-04-04T00:00:00+00:00https://channingwalton.github.io/posts/2023-04-04/ritronome<p><em>Ritronome</em> is an experimental metronome that can change tempo.</p>
<p>Conventional metronomes are very useful for all kinds of practice but they cannot be used for music that slows down or speeds up.
So I wrote <a href="/rit">Ritronome</a> to do exactly that and find out if having a variable metronome would actually help.</p>
<p>Go and try it out and let me know what you think on <a href="https://twitter.com/channingwalton">Twitter</a> or <a href="https://types.pl/@channingwalton">Mastodon</a>.</p>Channing WaltonRitronome is an experimental metronome that can change tempo.Complexity Trap2023-01-25T00:00:00+00:002023-01-25T00:00:00+00:00https://channingwalton.github.io/posts/2023-01-25/complexity-trap<p>There comes a point in complex domains where only developers have anything close to a complete understanding of the system behaviour. If you’re lucky.</p>
<p>I’ve worked on projects in finance and media that had reasonably complex behaviour. They start from basic requirements such as a simple, linear, workflow for bank workers to open accounts for new clients.</p>
<p>Then, in response to reasonable changes over time, things become complex. For example, the banker might need to ask the client questions so the application form is returned.</p>
<p>Later, the process needs to support multiple account holders so each account holder must complete an application form whilst the lead applicant completes more general details. All the forms must be collated and processed by the back office.</p>
<p>Then, automated identity checks are added to the process. They are asynchronous, taking minutes, so each relevant part of the application process for each account opener needs to be managed before the whole set of applications is sent for back office processing.</p>
<p>When the back office process involves multiple teams, with the potential for parts of applications to move between teams, things become even more complex.</p>
<p>That project had about 30 different stakeholder groups across the bank interested in different aspects of the system. <em>The only group that had the whole picture was the development team</em>. Analysts, managers, back office users, and other groups were unable to understand the whole system even though, collectively, they’d asked for all of it. And thats ok.</p>
<blockquote>
<p>Development is the crucible in which everything converges</p>
</blockquote>
<h2 id="was-it-wrong">Was it wrong?</h2>
<p>At every step in its evolution the system had features added for good reasons and it worked as intended. Everyone was happy because it automated a complex system that had been intensely manual. Accounts that took months to open were now opened in hours.</p>
<p>But, there was an uncomfortable accretion of complexity that didn’t exist with the manual process because it wasn’t feasible to manually do the things the system was doing.</p>
<p>Whilst the process was now rigorous and everything <em>could</em> be explained, users were nevertheless confused as they were seeing a small part of a much larger process.</p>
<p>It is difficult to know when complexity crosses the line where people, particularly non-devs, can comprehend everything. As developers we are able to look at code, technical docs, and tests, to help us. If users are not provided with similar tooling, which they won’t necessarily ask for, they will ask questions that developers will need to spend time answering which is not efficient for anyone.</p>
<p>We should be sensitive to increasing complexity and explore solutions to reduce it. That can be difficult with large systems built from complex requirements arising from the needs of many stakeholders, each of which may not fully appreciate the problems they’re creating for people who work directly with the system.</p>
<p>In our banking system, we ended up providing tools to help users see where applications were in the workflow and why, which was very helpful.</p>Channing WaltonThere comes a point in complex domains where only developers have anything close to a complete understanding of the system behaviour. If you’re lucky.Functor2022-08-29T00:00:00+00:002022-08-29T00:00:00+00:00https://channingwalton.github.io/posts/2022-08-29/functor<p>A non-mathematical introduction to functors.</p>
<p><code class="language-plaintext highlighter-rouge">Option</code> and <code class="language-plaintext highlighter-rouge">List</code> are two well-known classes, both of which have the function <code class="language-plaintext highlighter-rouge">map</code>
that takes a function that is applied to the values contained by the <code class="language-plaintext highlighter-rouge">List</code> or <code class="language-plaintext highlighter-rouge">Option</code>.</p>
<p>Here are two functions that take <code class="language-plaintext highlighter-rouge">List[Int]</code> and <code class="language-plaintext highlighter-rouge">Option[Int]</code> and increments the values:</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">inc</span><span class="o">(</span><span class="n">o</span><span class="k">:</span> <span class="kt">Option</span><span class="o">[</span><span class="kt">Int</span><span class="o">])</span><span class="k">:</span> <span class="kt">Option</span><span class="o">[</span><span class="kt">Int</span><span class="o">]</span> <span class="k">=</span> <span class="nv">o</span><span class="o">.</span><span class="py">map</span><span class="o">(</span><span class="n">i</span> <span class="k">=></span> <span class="n">i</span> <span class="o">+</span> <span class="mi">1</span><span class="o">)</span>
<span class="k">def</span> <span class="nf">inc</span><span class="o">(</span><span class="n">l</span><span class="k">:</span> <span class="kt">List</span><span class="o">[</span><span class="kt">Int</span><span class="o">])</span><span class="k">:</span> <span class="kt">List</span><span class="o">[</span><span class="kt">Int</span><span class="o">]</span> <span class="k">=</span> <span class="nv">l</span><span class="o">.</span><span class="py">map</span><span class="o">(</span><span class="n">i</span> <span class="k">=></span> <span class="n">i</span> <span class="o">+</span> <span class="mi">1</span><span class="o">)</span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">inc</code> functions do not care about <code class="language-plaintext highlighter-rouge">Option</code> or <code class="language-plaintext highlighter-rouge">List</code> particularly, they need a
way to work on the values they contain, but sadly <code class="language-plaintext highlighter-rouge">inc</code> is duplicated. It doesn’t have to be.</p>
<p>One way to fix this is to use a <code class="language-plaintext highlighter-rouge">Functor</code> that supplies a <code class="language-plaintext highlighter-rouge">map</code> function for any <code class="language-plaintext highlighter-rouge">F[_]</code>.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// scala 3</span>
<span class="k">trait</span> <span class="nc">Functor</span><span class="o">[</span><span class="kt">F</span><span class="o">[</span><span class="k">_</span><span class="o">]]</span> <span class="o">{</span>
<span class="n">extension</span> <span class="o">[</span><span class="kt">A</span>, <span class="kt">B</span><span class="o">](</span><span class="n">x</span><span class="k">:</span> <span class="kt">F</span><span class="o">[</span><span class="kt">A</span><span class="o">])</span>
<span class="k">def</span> <span class="nf">map</span><span class="o">(</span><span class="n">f</span><span class="k">:</span> <span class="kt">A</span> <span class="o">=></span> <span class="n">B</span><span class="o">)</span><span class="k">:</span> <span class="kt">F</span><span class="o">[</span><span class="kt">B</span><span class="o">]</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Given <code class="language-plaintext highlighter-rouge">x: F[A]</code>, <code class="language-plaintext highlighter-rouge">Functor[F]</code> provides a <code class="language-plaintext highlighter-rouge">map</code> method. Here is <code class="language-plaintext highlighter-rouge">inc</code> written using <code class="language-plaintext highlighter-rouge">Functor[F]</code>:</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">inc</span><span class="o">[</span><span class="kt">F</span><span class="o">[</span><span class="k">_</span><span class="o">]</span><span class="kt">:</span> <span class="kt">Functor</span><span class="o">](</span><span class="n">a</span><span class="k">:</span> <span class="kt">F</span><span class="o">[</span><span class="kt">Int</span><span class="o">])</span><span class="k">:</span> <span class="kt">F</span><span class="o">[</span><span class="kt">Int</span><span class="o">]</span> <span class="k">=</span> <span class="nv">a</span><span class="o">.</span><span class="py">map</span><span class="o">(</span><span class="n">i</span> <span class="k">=></span> <span class="n">i</span> <span class="o">+</span> <span class="mi">1</span><span class="o">)</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">F[_]: Functor</code> means that it requires an <em>instance</em> of <code class="language-plaintext highlighter-rouge">Functor[F]</code>.
Scala finds that instance by looking in <a href="https://docs.scala-lang.org/scala3/reference/changed-features/implicit-resolution.html#inner-main">lots of places</a>
for it, one being the companion object of the <code class="language-plaintext highlighter-rouge">F[_]</code>.</p>
<p>For <code class="language-plaintext highlighter-rouge">Option</code> and <code class="language-plaintext highlighter-rouge">List</code> those instances are:</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">given</span> <span class="nc">Functor</span><span class="o">[</span><span class="kt">Option</span><span class="o">]</span> <span class="k">with</span> <span class="o">{</span>
<span class="n">extension</span> <span class="o">[</span><span class="kt">A</span>, <span class="kt">B</span><span class="o">](</span><span class="n">m</span><span class="k">:</span> <span class="kt">Option</span><span class="o">[</span><span class="kt">A</span><span class="o">])</span>
<span class="k">override</span> <span class="k">def</span> <span class="nf">map</span><span class="o">(</span><span class="n">f</span><span class="k">:</span> <span class="kt">A</span> <span class="o">=></span> <span class="n">B</span><span class="o">)</span><span class="k">:</span> <span class="kt">Option</span><span class="o">[</span><span class="kt">B</span><span class="o">]</span> <span class="k">=</span> <span class="nv">m</span><span class="o">.</span><span class="py">map</span><span class="o">(</span><span class="n">f</span><span class="o">)</span>
<span class="o">}</span>
<span class="n">given</span> <span class="nc">Functor</span><span class="o">[</span><span class="kt">List</span><span class="o">]</span> <span class="k">with</span> <span class="o">{</span>
<span class="n">extension</span> <span class="o">[</span><span class="kt">A</span>, <span class="kt">B</span><span class="o">](</span><span class="n">m</span><span class="k">:</span> <span class="kt">List</span><span class="o">[</span><span class="kt">A</span><span class="o">])</span>
<span class="k">override</span> <span class="k">def</span> <span class="nf">map</span><span class="o">(</span><span class="n">f</span><span class="k">:</span> <span class="kt">A</span> <span class="o">=></span> <span class="n">B</span><span class="o">)</span><span class="k">:</span> <span class="kt">List</span><span class="o">[</span><span class="kt">B</span><span class="o">]</span> <span class="k">=</span> <span class="nv">m</span><span class="o">.</span><span class="py">map</span><span class="o">(</span><span class="n">f</span><span class="o">)</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Your own classes can play too:</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">final</span> <span class="k">case</span> <span class="k">class</span> <span class="nc">Blub</span><span class="o">[</span><span class="kt">A</span><span class="o">](</span><span class="n">v</span><span class="k">:</span> <span class="kt">A</span><span class="o">)</span>
<span class="k">object</span> <span class="nc">Blub</span> <span class="o">{</span>
<span class="n">given</span> <span class="nc">Functor</span><span class="o">[</span><span class="kt">Blub</span><span class="o">]</span> <span class="k">with</span> <span class="o">{</span>
<span class="n">extension</span> <span class="o">[</span><span class="kt">A</span>, <span class="kt">B</span><span class="o">](</span><span class="n">blub</span><span class="k">:</span> <span class="kt">Blub</span><span class="o">[</span><span class="kt">A</span><span class="o">])</span>
<span class="k">override</span> <span class="k">def</span> <span class="nf">map</span><span class="o">(</span><span class="n">f</span><span class="k">:</span> <span class="kt">A</span> <span class="o">=></span> <span class="n">B</span><span class="o">)</span><span class="k">:</span> <span class="kt">Blub</span><span class="o">[</span><span class="kt">B</span><span class="o">]</span>
<span class="k">=</span> <span class="nc">Blub</span><span class="o">(</span><span class="nf">f</span><span class="o">(</span><span class="nv">blub</span><span class="o">.</span><span class="py">v</span><span class="o">))</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="nf">inc</span><span class="o">(</span><span class="nc">Blub</span><span class="o">(</span><span class="mi">1</span><span class="o">))</span> <span class="n">gives</span> <span class="nc">Blub</span><span class="o">(</span><span class="mi">2</span><span class="o">)</span>
</code></pre></div></div>
<p>Note that this is an example of the typeclass pattern, read more <a href="https://docs.scala-lang.org/scala3/book/types-type-classes.html">here</a>.</p>
<h2 id="laws">Laws</h2>
<p>A proper functor must obey two laws:</p>
<ol>
<li><em>Identity</em>: Mapping with the <em>identity</em> function is a no-op</li>
<li><em>Composition</em>: <code class="language-plaintext highlighter-rouge">fa.map(f).map(g) = fa.map(f.andThen(g)</code></li>
</ol>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="https://typelevel.org/cats/typeclasses/functor.html">TypeLevel</a></li>
</ul>Channing WaltonA non-mathematical introduction to functors.Monads2022-08-29T00:00:00+00:002022-08-29T00:00:00+00:00https://channingwalton.github.io/posts/2022-08-29/monad<p>A non-mathematical introduction to Monads.</p>
<p>Read <a href="/posts/2022-08-29/Functors">Functor</a> first!</p>
<p>Code working with databases, performing HTTP requests, publishing messages on queues, all return something like <code class="language-plaintext highlighter-rouge">IO[A]</code> in functional codebases.</p>
<p>Here is a real-world example, slightly modified to protect the guilty, making calls to remote services and queues.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">completeSubscription</span><span class="o">(</span><span class="n">token</span><span class="k">:</span> <span class="kt">String</span><span class="o">,</span> <span class="n">id</span><span class="k">:</span> <span class="kt">UserId</span><span class="o">)</span><span class="k">:</span> <span class="kt">F</span><span class="o">[</span><span class="kt">Unit</span><span class="o">]</span> <span class="k">=</span>
<span class="k">for</span> <span class="o">{</span>
<span class="n">now</span> <span class="k"><-</span> <span class="nc">Clock</span><span class="o">[</span><span class="kt">F</span><span class="o">].</span><span class="py">instant</span> <span class="c1">// F[Instant]</span>
<span class="n">token</span> <span class="k"><-</span> <span class="nf">createToken</span><span class="o">(</span><span class="n">tokenString</span><span class="o">,</span> <span class="n">now</span><span class="o">)</span> <span class="c1">// F[Token]</span>
<span class="k">_</span> <span class="k"><-</span> <span class="nf">activate</span><span class="o">(</span><span class="n">now</span><span class="o">,</span> <span class="n">token</span><span class="o">,</span> <span class="n">id</span><span class="o">)</span> <span class="n">now</span><span class="o">)</span> <span class="c1">// F[Unit]</span>
<span class="k">_</span> <span class="k"><-</span> <span class="nf">entitleUser</span><span class="o">(</span><span class="n">id</span><span class="o">,</span> <span class="n">token</span><span class="o">)</span> <span class="c1">// F[Unit]</span>
<span class="k">_</span> <span class="k"><-</span> <span class="nf">sendSignupEmail</span><span class="o">(</span><span class="n">id</span><span class="o">,</span> <span class="n">token</span><span class="o">)</span> <span class="c1">// F[Unit]</span>
<span class="o">}</span> <span class="nf">yield</span> <span class="o">()</span>
</code></pre></div></div>
<p>Remember that a <code class="language-plaintext highlighter-rouge">for-comprehension</code> in Scala is just syntactic sugar for <code class="language-plaintext highlighter-rouge">map</code> and <code class="language-plaintext highlighter-rouge">flatMap</code>.</p>
<p>Here is a simpler example:</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">foo</span><span class="o">(</span><span class="n">x</span><span class="k">:</span> <span class="kt">F</span><span class="o">[</span><span class="kt">Int</span><span class="o">],</span> <span class="n">y</span><span class="k">:</span> <span class="kt">F</span><span class="o">[</span><span class="kt">Int</span><span class="o">])</span><span class="k">:</span> <span class="kt">F</span><span class="o">[</span><span class="kt">Int</span><span class="o">]</span> <span class="k">=</span>
<span class="k">for</span> <span class="o">{</span>
<span class="n">xv</span> <span class="k"><-</span> <span class="n">x</span>
<span class="n">yv</span> <span class="k"><-</span> <span class="n">y</span>
<span class="o">}</span> <span class="k">yield</span> <span class="n">xv</span> <span class="o">+</span> <span class="n">yv</span>
</code></pre></div></div>
<p>which desugars to:</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">sum</span><span class="o">(</span><span class="n">x</span><span class="k">:</span> <span class="kt">F</span><span class="o">[</span><span class="kt">Int</span><span class="o">],</span> <span class="n">y</span><span class="k">:</span> <span class="kt">F</span><span class="o">[</span><span class="kt">Int</span><span class="o">])</span><span class="k">:</span> <span class="kt">F</span><span class="o">[</span><span class="kt">Int</span><span class="o">]</span> <span class="k">=</span>
<span class="nv">x</span><span class="o">.</span><span class="py">flatMap</span><span class="o">(</span><span class="n">xv</span> <span class="k">=></span> <span class="nv">y</span><span class="o">.</span><span class="py">map</span><span class="o">(</span><span class="n">yv</span> <span class="k">=></span> <span class="n">xv</span> <span class="o">+</span> <span class="n">yv</span><span class="o">))</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">sum</code> and <code class="language-plaintext highlighter-rouge">completeSubscription</code> do not care about <code class="language-plaintext highlighter-rouge">F</code> itself, the so-called <em>effect</em>, they care about the values contained by it.
The <code class="language-plaintext highlighter-rouge">F[A]</code> returned from each of the generators in the <code class="language-plaintext highlighter-rouge">for-comprehension</code> must provide <code class="language-plaintext highlighter-rouge">flatMap</code> and <code class="language-plaintext highlighter-rouge">map</code>, it doesn’t need anything else.</p>
<p><code class="language-plaintext highlighter-rouge">Monads</code>, which are also <code class="language-plaintext highlighter-rouge">Functors</code>, are values that provide <code class="language-plaintext highlighter-rouge">map</code> and <code class="language-plaintext highlighter-rouge">flatMap</code> for an instance of an effect, <code class="language-plaintext highlighter-rouge">F[_]</code>.
Here is <code class="language-plaintext highlighter-rouge">sum</code> rewritten using a <code class="language-plaintext highlighter-rouge">Monad</code> <em>for any <code class="language-plaintext highlighter-rouge">F[_]</code></em> that has a <code class="language-plaintext highlighter-rouge">Monad[F]</code>:</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">sum</span><span class="o">[</span><span class="kt">F</span><span class="o">[</span><span class="k">_</span><span class="o">]](</span><span class="n">x</span><span class="k">:</span> <span class="kt">F</span><span class="o">[</span><span class="kt">Int</span><span class="o">],</span> <span class="n">y</span><span class="k">:</span> <span class="kt">F</span><span class="o">[</span><span class="kt">Int</span><span class="o">])(</span><span class="n">using</span> <span class="nc">Monad</span><span class="o">[</span><span class="kt">F</span><span class="o">])</span><span class="k">:</span> <span class="kt">F</span><span class="o">[</span><span class="kt">Int</span><span class="o">]</span> <span class="k">=</span>
<span class="k">for</span> <span class="o">{</span>
<span class="n">xv</span> <span class="k"><-</span> <span class="n">x</span>
<span class="n">yv</span> <span class="k"><-</span> <span class="n">y</span>
<span class="o">}</span> <span class="k">yield</span> <span class="n">xv</span> <span class="o">+</span> <span class="n">yv</span>
</code></pre></div></div>
<h2 id="writing-a-monad">Writing a Monad</h2>
<p>A <code class="language-plaintext highlighter-rouge">Monad</code> is a <code class="language-plaintext highlighter-rouge">Functor</code> and also provides the function described above for some <code class="language-plaintext highlighter-rouge">F[_]</code>:</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">trait</span> <span class="nc">Monad</span><span class="o">[</span><span class="kt">F</span><span class="o">[</span><span class="k">_</span><span class="o">]]</span> <span class="nc">extends</span> <span class="nc">Functor</span><span class="o">[</span><span class="kt">F</span><span class="o">]</span> <span class="o">{</span>
<span class="n">extension</span> <span class="o">[</span><span class="kt">A</span>, <span class="kt">B</span><span class="o">](</span><span class="n">x</span><span class="k">:</span> <span class="kt">F</span><span class="o">[</span><span class="kt">A</span><span class="o">])</span>
<span class="k">def</span> <span class="nf">flatMap</span><span class="o">(</span><span class="n">f</span><span class="k">:</span> <span class="kt">A</span> <span class="o">=></span> <span class="n">F</span><span class="o">[</span><span class="kt">B</span><span class="o">])</span><span class="k">:</span> <span class="kt">F</span><span class="o">[</span><span class="kt">B</span><span class="o">]</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Here are example instances for <code class="language-plaintext highlighter-rouge">Option</code> and <code class="language-plaintext highlighter-rouge">List</code>:</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">given</span> <span class="nc">Monad</span><span class="o">[</span><span class="kt">Option</span><span class="o">]</span> <span class="k">with</span> <span class="o">{</span>
<span class="n">extension</span> <span class="o">[</span><span class="kt">A</span>, <span class="kt">B</span><span class="o">](</span><span class="n">m</span><span class="k">:</span> <span class="kt">Option</span><span class="o">[</span><span class="kt">A</span><span class="o">])</span>
<span class="k">override</span> <span class="k">def</span> <span class="nf">map</span><span class="o">(</span><span class="n">f</span><span class="k">:</span> <span class="kt">A</span> <span class="o">=></span> <span class="n">B</span><span class="o">)</span><span class="k">:</span> <span class="kt">Option</span><span class="o">[</span><span class="kt">B</span><span class="o">]</span> <span class="k">=</span> <span class="nv">m</span><span class="o">.</span><span class="py">map</span><span class="o">(</span><span class="n">f</span><span class="o">)</span>
<span class="k">override</span> <span class="k">def</span> <span class="nf">flatMap</span><span class="o">(</span><span class="n">f</span><span class="k">:</span> <span class="kt">A</span> <span class="o">=></span> <span class="nc">Option</span><span class="o">[</span><span class="kt">B</span><span class="o">])</span><span class="k">:</span> <span class="kt">Option</span><span class="o">[</span><span class="kt">B</span><span class="o">]</span> <span class="k">=</span> <span class="nv">m</span><span class="o">.</span><span class="py">flatMap</span><span class="o">(</span><span class="n">f</span><span class="o">)</span>
<span class="o">}</span>
<span class="n">given</span> <span class="nc">Monad</span><span class="o">[</span><span class="kt">List</span><span class="o">]</span> <span class="k">with</span> <span class="o">{</span>
<span class="n">extension</span> <span class="o">[</span><span class="kt">A</span>, <span class="kt">B</span><span class="o">](</span><span class="n">m</span><span class="k">:</span> <span class="kt">List</span><span class="o">[</span><span class="kt">A</span><span class="o">])</span>
<span class="k">override</span> <span class="k">def</span> <span class="nf">map</span><span class="o">(</span><span class="n">f</span><span class="k">:</span> <span class="kt">A</span> <span class="o">=></span> <span class="n">B</span><span class="o">)</span><span class="k">:</span> <span class="kt">List</span><span class="o">[</span><span class="kt">B</span><span class="o">]</span> <span class="k">=</span> <span class="nv">m</span><span class="o">.</span><span class="py">map</span><span class="o">(</span><span class="n">f</span><span class="o">)</span>
<span class="k">override</span> <span class="k">def</span> <span class="nf">map</span><span class="o">(</span><span class="n">f</span><span class="k">:</span> <span class="kt">A</span> <span class="o">=></span> <span class="nc">List</span><span class="o">[</span><span class="kt">B</span><span class="o">])</span><span class="k">:</span> <span class="kt">List</span><span class="o">[</span><span class="kt">B</span><span class="o">]</span> <span class="k">=</span> <span class="nv">m</span><span class="o">.</span><span class="py">flatMap</span><span class="o">(</span><span class="n">f</span><span class="o">)</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Remember that Scala’s <code class="language-plaintext highlighter-rouge">List</code> and <code class="language-plaintext highlighter-rouge">Option</code> have <code class="language-plaintext highlighter-rouge">flatMap</code> functions already,
but don’t confuse that with the Monad implementations.</p>
<p>Your own types can play too:</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">final</span> <span class="k">case</span> <span class="k">class</span> <span class="nc">Blub</span><span class="o">[</span><span class="kt">A</span><span class="o">](</span><span class="n">v</span><span class="k">:</span> <span class="kt">A</span><span class="o">)</span>
<span class="k">object</span> <span class="nc">Blub</span> <span class="o">{</span>
<span class="n">given</span> <span class="nc">Monad</span><span class="o">[</span><span class="kt">Blub</span><span class="o">]</span> <span class="k">with</span> <span class="o">{</span>
<span class="n">extension</span> <span class="o">[</span><span class="kt">A</span>, <span class="kt">B</span><span class="o">](</span><span class="n">blub</span><span class="k">:</span> <span class="kt">Blub</span><span class="o">[</span><span class="kt">A</span><span class="o">])</span>
<span class="k">override</span> <span class="k">def</span> <span class="nf">flatMap</span><span class="o">(</span><span class="n">f</span><span class="k">:</span> <span class="kt">A</span> <span class="o">=></span> <span class="nc">Blub</span><span class="o">[</span><span class="kt">B</span><span class="o">])</span><span class="k">:</span> <span class="kt">Blub</span><span class="o">[</span><span class="kt">B</span><span class="o">]</span> <span class="k">=</span> <span class="nf">f</span><span class="o">(</span><span class="nv">blub</span><span class="o">.</span><span class="py">v</span><span class="o">)</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="nf">sum</span><span class="o">(</span><span class="nc">Blub</span><span class="o">(</span><span class="mi">1</span><span class="o">),</span> <span class="nc">Blub</span><span class="o">(</span><span class="mi">2</span><span class="o">))</span> <span class="c1">// Blub(3)</span>
</code></pre></div></div>
<h2 id="more">More</h2>
<ol>
<li>A Monad is also a <a href="/posts/2022-08-29/Functors">Functor</a></li>
<li>Monad has three laws:
<ol>
<li>left identity: <code class="language-plaintext highlighter-rouge">(Monad[F].pure(x).flatMap(f)) === f(x)</code></li>
<li>right identity: <code class="language-plaintext highlighter-rouge">(m.flatMap(Monad[F].pure(_))) === m</code></li>
<li>associativity: <code class="language-plaintext highlighter-rouge">(m.flatMap(f)).flatMap(g) === m.flatMap(x => f(x).flatMap(g))</code></li>
</ol>
</li>
<li>Learn about this in <a href="https://adit.io/posts/2013-04-17-functors,_applicatives,_and_monads_in_pictures.html">pictures</a> (Haskell)</li>
<li><a href="http://tomasp.net/academic/papers/monads/monads-programming.pdf">What we talk about when we talk about monads.pdf</a></li>
</ol>Channing WaltonA non-mathematical introduction to Monads.Make Building Blocks2021-05-18T10:00:00+00:002021-05-18T10:00:00+00:00https://channingwalton.github.io/posts/2020-05-18/make-building-blocks<p>How do we write software that is simple, extensible, composeable? This is a very difficult question to answer.</p>
<p>Good software has <em>Quality Without a Name</em>, a phrase coined by Christopher Alexander in his book
<em>The Timeless Way of Building</em>, a quality that is hard to create and describe but we all know it when we see it.</p>
<p>The best we can do is to find heuristics which help us produce simple, beautiful software, and here
I discuss what I believe to be the most fundamental step which is so often not done.</p>
<h2 id="axioms">Axioms</h2>
<ol>
<li>A well designed system is built from well designed components built from smaller well designed components built from well designed elements such as functions, data types, objects, etc. The paradigm – functional programming, object orientation, etc., really doesn’t matter, they all offer the means to write well designed systems.</li>
<li>Well designed components, compose well.</li>
<li>For composition to be possible, composeable elements must exist.</li>
<li>For a thing to be quickly and easily understood by a human, it must be comprised of a small number of elements, six at most for the average mind. With experience we are able to chunk information and hold more information, but we are still limited to six chunks.</li>
</ol>
<h2 id="decomposition">Decomposition</h2>
<h3 id="new-code">New Code</h3>
<p><a href="https://mitpress.mit.edu/sites/default/files/sicp/full-text/sicp/book/node28.html">Programming by wishful thinking</a>
is a good technique to produce code that is made from lots of small parts, from the classic book,
<a href="https://mitpress.mit.edu/sites/default/files/sicp/full-text/book/book.html">SICP</a> by Harold Abelson and Gerald Sussman.</p>
<p>Instead of writing code in a depth first traversal, which is error prone, stressful, and frustrating,
write code assuming the functions you need already exist. This means that you are only having to
think at one level of abstraction, since anything you need at a lower level <em>already exists</em>, in the future.</p>
<p>Also, you’re likely to write code that resembles a description of the algorithm:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">makeTea</span>
<span class="n">hotWater</span> <span class="o">=</span> <span class="n">getHotWater</span>
<span class="n">pot</span> <span class="o">=</span> <span class="n">getPot</span>
<span class="n">tea</span> <span class="o">=</span> <span class="n">getTea</span>
<span class="n">pot</span><span class="p">.</span><span class="k">with</span><span class="p">(</span><span class="n">water</span><span class="p">).</span><span class="k">with</span><span class="p">(</span><span class="n">tea</span><span class="p">)</span>
<span class="n">end</span>
</code></pre></div></div>
<h3 id="existing-code">Existing Code</h3>
<p>The most fundamental step to satisfy the axioms is simply to <em>extract</em>. The parts of methods,
functions, procedures, expressions, and so on, should be extracted out until those functions
are <a href="https://twitter.com/deusaquilus/status/1394011278843199491?s=20">tiny</a> – one
expression or one control structure in a function or method.</p>
<p>Larger scale structures such as classes, packages, or modules, are not of concern yet, all
that matters is the work of pulling the code apart at the lowest level until everything is
laid out before you, like pieces of lego.</p>
<h2 id="composition">Composition</h2>
<p>The result of both the approaches described will be many small functions around the
places they were extracted from, which may not be very useful in general,
but there are important consequences.</p>
<p>First, people reading the code will have an easier time understanding those smaller functions.
Of course, readers still need to understand the whole program, but they will
find it a lot easier to do that if the program is composed of small parts.</p>
<p>Second, having lots of small functions enables the programmer to stand back and
form a bigger picture. They might notice duplication, or groups of functions
that relate to each other, or remember other functions in other places that look
the same. They’re now able to start building structures at a higher level of
abstraction. Without those small functions they could not do that.</p>
<h2 id="thoughts">Thoughts</h2>
<p>This seems too simple, but it is a prerequisite to being able to understand what
the code is built <em>of</em>. The programmer still needs to make good choices about
how to compose those functions, but at least they have the possibility to do it
now. Without decomposed functions, there can be no composition.</p>Channing WaltonHow do we write software that is simple, extensible, composeable? This is a very difficult question to answer.Playing the Piano with Ease2021-03-13T14:00:00+00:002021-03-13T14:00:00+00:00https://channingwalton.github.io/posts/2020-03-13/playing-with-ease<p>How hard can it be?</p>
<p>I’ve been studying the piano for most of my life. I was first taught by my mother, a classically trained professional musician who also ran her own teaching studio. But I stopped learning in my 20s and restarted a few years ago, twenty or so years later.</p>
<p>I’ve learned a huge amount from pianists like <a href="https://www.youtube.com/user/joshwrightpiano">Josh Wright</a>,
<a href="https://www.youtube.com/user/PaulBartonPiano">Paul Barton</a>, <a href="https://www.youtube.com/user/cedarvillemusic">John Mortensen</a>, and many others.</p>
<p>The fundamentals of piano technique seem pretty straight forward: just keep completely relaxed, calm, serene, maintain good posture if you want your back to survive, and keep doing all that when playing at <a href="https://www.youtube.com/watch?v=ZZyokBXgEJY">ridiculous speeds</a>.</p>
<p>Playing with complete ease is very difficult to master. How does Horowitz do <a href="https://www.youtube.com/watch?v=FxhbAGwEYGQ">this</a>, he seems to be petting the piano and the piano plays itself?</p>
<p>In trying to achieve this elusive quality of complete ease my big mistake has been to assume that <em>ease</em> will emerge from lots and lots of hard, technical practice.</p>
<p>But I was wrong — at least for me. I got this completely backwards. My recent experience of flipping things around are proving how badly I got this wrong. Simply put, <strong>start</strong> with being at ease and relaxed and then build everything else on top of that foundation. Ease cannot be added afterwards.</p>
<p>At this point many teachers will be screaming, “Of course!!!”. But, somehow I didn’t really get it, and I’ve been told it in lots of ways, but it has always been indirect: “slow down”, “relax”, “curl your fingers”, “don’t tense up”.</p>
<p>I needed to start from being at ease as a foundational principle. What does that mean?</p>
<ol>
<li>First, are you <em>sitting</em> comfortably? Is you back straight? Can you sit like that for some time, after all thats what you are going to be doing. If you can’t sit on the chair comfortably then things aren’t going to go well. Make sure the <a href="https://youtu.be/5YlKWaPxnj8">bench position</a> is good.</li>
<li>Now rest your hands on the keyboard. Are you still comfortable? Are your shoulders and body relaxed? Are you still sitting comfortably? Is your chair still at the <a href="https://youtu.be/5YlKWaPxnj8">right height</a>.</li>
<li>Are your hands touching the keyboard with a good shape? (See <a href="https://www.youtube.com/watch?v=r_PPWiTEdNA">this</a> or <a href="https://www.youtube.com/watch?v=7s4V98-lElk">this</a>)</li>
<li>Are you still comfortable? Remove all tension by adjusting your position.</li>
<li>Play a note, a five finger scale (C-G), a chord. Are you <em>still comfortable</em> and relaxed. Are you using the lightest touch you can? Is what you’re playing sounding even, or beautiful?</li>
<li>Do not go beyond this point until you can play something very simple, slowly and completely at ease.</li>
</ol>
<p><em>You need to learn what it feels like to play with complete ease</em> by playing something so simple (a five note scale) that you can think about your body and not what you’re playing. That feeling is what you need to strive for in all your playing, but first you need to know what that feeling is. Once you’ve felt it you’ll never forget it, its almost euphoric.</p>
<p>In all your practice, that is the state you need to achieve <em>first</em>. If you deviate from it then reset, <strong>slow down</strong>, and find it again.</p>
<p>Fundamentally, I don’t think its possible to develop playing at ease as an afterthought, it doesn’t emerge from playing lots of technical exercises or when you <em>get good enough</em>. Its an upfront <em>decision</em> that everything else rests upon.</p>
<p>The work of a lifetime is to learn how to keep yourself in that state whilst playing more and more difficult music.</p>
<p>ps. My brother, Lance, tells me that this is what the <a href="https://en.wikipedia.org/wiki/Theodor_Leschetizky">Leschetizky Method</a> espouses. I am very happy that I’m in good company.</p>
<hr />Channing WaltonHow hard can it be?Evergreen Software Documentation2021-02-20T21:30:30+00:002021-02-20T21:30:30+00:00https://channingwalton.github.io/posts/2020-02-20/evergreen-software-documentation<p>Software Documentation is hard to do, tedious to maintain, unsatisfactory, and always wrong.</p>
<p>The problem is that the information is both a <a href="https://en.wikipedia.org/wiki/Heterarchy">heterarchy</a> and kind of a <a href="https://en.wikipedia.org/wiki/Hermeneutic_circle">hermeneutic circle</a>, which is why managing that knowledge with traditional tools — Word, Powerpoint, Readme’s — is so hard. Various document management systems are available but they don’t suit everyone, particular software developers, and just raise barriers to people getting their thoughts down.</p>
<p>The result is that documentation just isn’t done, or at best a few documents are produced from different points of view for different audiences — APIs for developers, powerpoint for management, etc., but nothing really captures everything or provides any means of discovery, learning, or synthesis of new ideas.</p>
<p>But perhaps we can improve on this by drawing on the experience of knowledge management and in particular from systems like <a href="https://andymatuschak.org">Andy Matuschak’s</a> <a href="https://notes.andymatuschak.org/Evergreen_notes">Evergreen Notes</a>, <a href="https://notes.linkingyourthinking.com/Home">Linking Your Thinking</a> and <a href="https://en.wikipedia.org/wiki/Niklas_Luhmann">Niklas Luhman’s</a> <a href="https://zettelkasten.de/posts/overview/">Zettelkasten</a> system.</p>
<h2 id="what-was-that">What was that?</h2>
<p>A quick aside — the <a href="https://notes.linkingyourthinking.com/Home">Linking Your Thinking</a> website looks interesting, you might be wondering what that was. It was published from <a href="https://obsidian.md">Obsidian.md</a>, a tool that enables everything I’m about to discuss, but from a personal point of view. Documents are written in markdown, linked simply like a wiki, with tags, and importantly, automatic discovery of missing links, and an <a href="https://pbs.twimg.com/media/EjHkSkuXYAM3o9_?format=jpg&name=large">amazing graph</a> of all your notes.</p>
<p>Back to the point …</p>
<h2 id="evergreen-notes">Evergreen Notes</h2>
<p>Andy Matuschak describes his notes in detail on his <a href="https://notes.andymatuschak.org/Evergreen_notes">website</a>, which is itself an example of evergreen notes. The summary is:</p>
<ul>
<li><a href="https://notes.andymatuschak.org/z4Rrmh17vMBbauEGnFPTZSK3UmdsGExLRfZz1">Evergreen notes should be atomic</a></li>
<li><a href="https://notes.andymatuschak.org/z6bci25mVUBNFdVWSrQNKr6u7AZ1jFzfTVbMF">Evergreen notes should be concept-oriented</a></li>
<li><a href="https://notes.andymatuschak.org/z2HUE4ABbQjUNjrNemvkTCsLa1LPDRuwh1tXC">Evergreen notes should be densely linked</a></li>
<li><a href="https://notes.andymatuschak.org/zg3fYweZpbHeBTpcYke5mF4ZfrJutYcQEtFo">Write about what you read</a></li>
<li><a href="https://notes.andymatuschak.org/z3N113rxPFreW9xUkLkUFomr2LUqfXbdCo3M">Use a reading inbox to capture possibly-useful references</a></li>
</ul>
<h2 id="organisation-an-inbox-and-permanent-notes">Organisation: an Inbox and Permanent Notes</h2>
<p>There are two main places for notes: the inbox and permanent notes.</p>
<p>The inbox is important, it allows you to throw ideas into it as they cross your mind, freeing you to return to it later. The <em>whole</em> point is to minimise the impedance to writing.</p>
<p>Moving notes from the inbox to the permanent <em>evergreen forest</em> of notes can be done at leisure, and its where the fun is — add ideas to existing notes, create new ones, link notes, etc.</p>
<p>But don’t forget the simple rules, particularly that notes should be atomic. If they start getting too big, break them up and link them.</p>
<p>Evergreen notes live in a flat structure, not directories, because we want to keep things simple, but having a densely linked set of atomic notes in a flat structure isn’t enough.</p>
<p>We need a way to navigate the forest …</p>
<h2 id="maps-of-content">Maps of Content</h2>
<p><a href="https://notes.linkingyourthinking.com/Home">Linking Your Thinking</a> introduces the idea of <em>Maps of Content</em> (MOCs) to solve the problem of becoming lost in the forest. I suggest you <a href="https://publish.obsidian.md/lyt-kit/MOCs+Overview">read</a> their site about MOCs as there is no point reproducing that here.</p>
<p><strong>TLDR;</strong> MOCs are notes that collate other notes by linking to them, possibly with a narrative, but they are more than just a table of content. Tables of content are rigid, but MOCs link to notes and add their own spin, they sit above the forest of notes, forming a second layer of abstraction and can be notes themselves.</p>
<p>For personal knowledge management systems MOCs might just be simple collections of links to notes, but for software documentation they could be used as a means of a progressively summarising the detailed notes for different purposes: architecture, design overview, etc., — <em>MOCs all the way up</em>.</p>
<p>I find the idea of MOCs very liberating as it unburdens you from having to think about a conventional document structure in which information has to reside, locked away in one place.</p>
<p>Notes are free, MOCs independently provide structure.</p>
<h2 id="evergreen-software-documentation">Evergreen Software Documentation</h2>
<p>The proposition is to use evergreen notes and LYT’s MOCs to document software systems and everything related to them.</p>
<p><strong>Easy to write:</strong> The fundamental atomic notes are easy to write because they are small, focussed, and enable people to write what they are thinking without forcing them to think about which document(s) their important thought should live in — which stops everyone from writing anything — if they can find it!</p>
<p><strong>Write documents with ease:</strong> Something that has been discovered by researchers and writers using these techniques is that <em>when you have a richly interconnected set of notes, writing becomes very easy</em>. You never have to start writing from a blank page as there is a wealth of information to collate from in the form of your notes. The first draft is already written!</p>
<p>It has been said the problem now is having <em>too much</em> to write about rather than too little.</p>
<p><strong>External documents:</strong> External documents managed in other systems is inevitable. I suggest that any external document has a proxy note so that it can be referenced easily.</p>
<p><strong>Evergreen:</strong> Notes are easy to keep up to date, as they are small, discrete, focussed, searchable, plaintext, and easy to modify.</p>
<p><strong>Discoverable:</strong> Tools can assist in finding relationships between notes. For example, when a document’s name appears in another document but not actually linked a tool such as <a href="https://publish.obsidian.md/help/How+to/Working+with+backlinks">Obsidian’s Unlinked Mentions</a> can see that and suggest adding a link.</p>
<p>Surprising ideas can emerge as relationships between notes emerge.</p>
<p><strong>For everyone:</strong> A wiki could form the basis of this system so that everyone can contribute, but a better idea might be to simply put the notes in a git repository and use Obisidian to work with them. That may be a bit technical for non-techies, but dealing with merging isn’t difficult for plain text, and there are <a href="https://github.com/denolehov/obsidian-git">plugins</a> for Obsidian to automatically sync with a git repo.</p>
<p><strong>Scary:</strong> To some, abandoning preconceived notions of structure is scary and so they might resist, preferring to continue with their own system. But I suspect the advantages will outweigh any concerns pretty quickly.</p>
<p><strong>Tool Support:</strong> It is necessary to use something like a Wiki or Obisidian to facilitate discovery and navigation. If the notes are in a shared repository then anyone can check them out and use Obisidian to work with the Notes as all it needs is a folder of markdown documents.</p>
<p><strong>The forest is where you think:</strong> Having a searchable, interconnected system of notes forms a second brain and a place to help you think and discover new information or ideas, a critical result of Luhman’s Zettelkasten and the reason he gave for his enormous productivity (70 books, 400 articles). With the complexity of software it would be fascinating to see what emerges. (See <a href="http://luhmann.surge.sh/communicating-with-slip-boxes">Luhman on thinking with slip boxes</a>.)</p>
<p><strong>Its all about the maps:</strong> MOCs are where notes are brought together to form a narrative for a particular audience, drawing on external documents if necessary, but try to keep information in the notes. The MOCs themselves are just notes and can easily be linked to from another MOC or note.</p>
<h2 id="johnny-decimal">Johnny Decimal</h2>
<p>If having absolutely no structure for your permanent notes sounds a bit crazy, you could consider using the <a href="https://johnnydecimal.com">Johnny Decimal</a> system.</p>
<p><strong>TLDR;</strong> arrange content in at most ten folders called <em>areas</em>, with at most ten sub-folders called <em>categories</em>, which is enough for anything mainly because people can’t cope well beyond that.</p>
<p>Each area and category is prefixed with a two digit number, and notes are placed in categories prefixed by their category number followed by an incrementing number:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>10-19 Finance
11 Tax Returns
12 Payroll
12.01 Staff bank details
12.02 Payroll Spreadsheet
…
…
20-29 Administration
21 Company
22 Contracts
…
</code></pre></div></div>
<p>The idea is that people will get used to the numbers pretty quickly and so have a shorthand to refer to things.</p>
<p>I don’t like the numbering part of this idea as it seems to be adding something that isn’t really needed, but I thought I’d put it out there, someone might find it useful.</p>
<h2 id="final-thoughts">Final Thoughts</h2>
<p>There is a great deal that software people could learn about managing complex systems of information from other disciplines. We are very good at creating complexity, just not very good at managing it.</p>
<p>Reducing the impedance to writing documentation in the first place is critical! Just knowing there is a simple place to put ideas or information greatly increases the ease with which people can write.</p>
<p>Evergreen notes is a simple concept, which is very important as we don’t want complexity to get in the way of us writing. Keep the notes focussed on a single idea, link and tag them.</p>
<p>MOCs enable us to navigate the notes, they are maps offering a way of representing the notes in different ways, from different points of view, flexibly and independent of the notes themselves. Contrast this with a more traditional document approach where information is locked into a single view and has to be copied when its needed again.</p>
<p>Using a system like this for software would be an interesting experiment, I don’t think anyone has tried doing this kind of thing collectively but I’d be happy to learn others have had this idea before me — I’d be astonised if someone hadn’t already.</p>
<h2 id="interesting-stuff">Interesting Stuff</h2>
<ul>
<li><a href="https://notes.andymatuschak.org/Evergreen_notes">Evergreen Notes</a></li>
<li><a href="https://publish.obsidian.md/lyt-kit/_Start+Here">Linking Your Thinking</a></li>
<li><a href="https://zettelkasten.de/posts/overview/">Zettelkasten Overview</a></li>
<li><a href="http://luhmann.surge.sh/communicating-with-slip-boxes">Luhman on thinking with slip boxes</a></li>
<li>Sönke Ahrens’s <a href="https://smile.amazon.co.uk/How-Take-Smart-Notes-Nonfiction/dp/1542866502/">How to take Smart Notes</a> and his <a href="https://www.youtube.com/watch?v=nPOI4f7yCag">video</a></li>
<li><a href="https://www.youtube.com/watch?v=dddIzjZ_WUo">Creating the Brainforest</a></li>
</ul>
<hr />Channing WaltonSoftware Documentation is hard to do, tedious to maintain, unsatisfactory, and always wrong.