How I Auto-Generate Books and Links Posts

I’m assuming you have a cool tool that keeps track of these things when you read them? And that you’ve already done a post on it (which I somehow can’t find)?

— Katie Ford (@classickatie)

Well, I did mention it briefly I think, but I've never described it in detail.

a file in my dropbox every time I favorite, highlight, or comment an article in Instapaper. I run a script on my computer to scoop up the links for the month, and format them for a blog post.

Before

I used to have all my favorite sites in a giant folder in my Firefox bookmarks. After I discovered the wonders of Google Reader (RIP!), I moved most of those self-updating blogs and sites into my RSS reader... but I was left with an archive of Neat Things I Think May Be Useful in that folder. I eventually moved that into the wonderful delicious.com (RIP!) until it's sudden but inevitable demise.

I also at some point set up some kind of integration where I automatically posted everything I starred to twitter. That explains my ridiculously high tweet count, and for anyone who was following me at that time... I am deeply sorry.

Over the past few years, I've used Instapaper to save and read all incoming links. I'd comb through social media and my RSS reader, anything that caught my attention would go to instapaper where I could read the chosen few.

At some point, I realized blowing up twitter with everything I thought was interesting was probably frustrating. I unfollow people who do that, why would I do that?

I also didn't have a long term place to store This Might Be Useful Someday links, so I was just using folders in Instapaper, which worked well until they launched their premium pricing which took away search and unlimited highlights.¹

Now

When Instapaper changed their pricing, I just really gotten into their highlight system. It made finding the quote or part of the page easier to find. I had also started reading about zettelkasten, and it fit the need well. I also had been slowly trying to wean my knowledge system from web-services and move it to plain text I could own and take with me wherever I go... so I whipped up a simple ifttt.com applet:

The result is a giant folder of everything I've liked in Instapaper!

As a developer, I have tons of tools for filtering, finding, and relating words in plain text. If I'm going to start researching about starting a design system at work, I can rg the whole folder for any files that contain the words "design system." I can further filter to files also containing the words "tool" if I'm looking for tooling to help. It's been very powerful.

The zettelkasten method involves both recording the quote you found useful, and the thought that the quote/content inspired in your mind. Recording the inspiration is the key thought, even if it's as simple as "I can use x to do y later." If nothing else, it's more key words for your full-text search. An example:

Daniel Gross on productivity - Marginal REVOLUTION

> The longer you think about a task without doing it, the less novel it becomes
> to do [...] my solution is to (somewhat counter-intuitively) not think about
> the task until I am ready to fully execute it. I do not unwrap the piece of
> gum until I’m ready to enjoy it in its entirety. I need to save the fun of
> thinking to pull myself into flow.

This is a very interesting idea. I can see the value of it, especially as I
over-engineer a todo task.

January 28, 2020 at 01:07PM
via Instapaper https://marginalrevolution.com/marginalrevolution/2020/01/daniel-gross-on-productivity.html

At some point I started blogging again, and inspired by Gruber and others decided to start putting links and commentary on my blog. The first was built manually... but I quickly started writing a script. The descriptions I had been adding to each link provided the commentary that I needed to make the linkpost be a little bit more meaningful than just "I read this. Here."

When it's the end of the month, I run bundle exec middleman article "Books and Links: February" to generate a blog post. I then edit it in vim, running read !elixir get_links.ex, thus pulling the result of the script into the file.

Don't laugh, it's overly complicated. Here's a tour:

defmodule GetLinks do
  @moduledoc false

  # <... abbreviated ...>

  def run(month) do
    """
    ---

    title: "Books and Links: <month>"
    date:
    tags: reading, books, links

    ---

    """ |> IO.write

Middleman relies on YAML frontmatter to drive some of the features. I print some out for consistency.

    IO.write("## Books\n\n")
    with :ok <- File.cd(Path.expand("~/dropbox/wiki/booknotes/2020")) do
      File.ls!()
      |> Enum.filter(& &1 =~ ~r/.txt|.md/)
      |> Enum.filter(&is_current_month?(&1, month))
    end
    |> Enum.sort_by(&File.stat!(&1, time: :local))
    |> Enum.map(&print_reference/1)
    |> Enum.join
    |> IO.write

I also make reviews for books that I read... there's another whole workflow for recording that that has been updated since that post. This snippet just dumps them under the right heading. I usually edit out most of the notes and leave the high-level review at the top.

    IO.write("## Links\n\n")
    with :ok <- File.cd(Path.expand("~/dropbox/wiki/links")) do
      File.ls!()
      |> Enum.filter(& &1 =~ ~r/.txt/)
      |> Enum.filter(&is_current_month?(&1, month))
    end
    |> Enum.sort_by(&File.stat!(&1, time: :local))
    |> Enum.map(&print_reference/1)
    |> Enum.join
    |> IO.write
  end

This does the same thing for the links.

There's some more I've concealed (although the full script is linked below,) but the bulk of the script is extracting the title, content, and link, and printing it into a markdown format, like this:

  def build_markdown(%{link: _} = obj) do
    """
    <h3>
      <a href='#{obj.link}'>#{obj.title}</a>
    </h3>

    #{obj.content}

    [Read more...](#{obj.link})

    """
  end

While writing this post, I'm seeing a good possibilty of a refactor ahead, especially with the hardcoded year in the books section, but that can wait.

These days, I just read on instapaper. If I find something I want to keep, I favorite it. A few moments later, I'll edit the file (often in the Dropbox.app on my iPhone,) adding some commentary or quotes that I like.

At the end of the month, I'll run this script, and then edit the resulting file. I'll cut out some links that are redundant or I can't picture anyone else wanting, add some images or videos if relevant, rewrite the booknotes for public use, and push to S3.

I'd like to automate this further, letting the site self-publish on git push, but that can come later.

References

• The full get_links.ex script.