Sacha Chua - tag - nodejs

Listing random npmjs.com packages updated today

2016-02-21T07:20:58Z

I was looking for a way to randomly learn about packages hosted at npmjs.com so that I can come across libraries I might not have thought of searching for. The registry data is available at https://registry.npmjs.org/, and there's a public CouchDB mirror at https://skimdb.npmjs.com/registry . Someday, when I know more about CouchDB, I might be able to query it and do other things.

In the meantime, this Github issue pointed me to a view of all packages modified today, which is a good-enough proxy for what I'm interested in.

Here's an AngularJS app that displays the list and highlights a random item.

<html ng-app="myApp">
  <head>
    <script type="text/javascript"
      src="https://cdnjs.cloudflare.com/ajax/libs/angular.js/1.5.0/angular.min.js">script>
    <script>
     // https://registry.npmjs.org/-/all/static/today.json
     // from https://github.com/npm/npm-registry-couchapp/issues/242
     var app = angular.module('myApp', []);
     app.controller('npmTodayCtrl', function($scope, $http) {
       $scope.randomize = function() {
         $scope.random = $scope.packages[Math.floor(Math.random() * $scope.packages.length)];
       }
       $http.get('https://registry.npmjs.org/-/all/static/today.json').then(function(info) {
         $scope.packages = info.data;
         $scope.randomize();
       });
     });
    script>
  head>
  <body ng-controller="npmTodayCtrl">
    <div><a href="" ng-click="randomize()">Random highlight:a>div>
    <div ng-if="random" style="margin-top: 1em; font-size: x-large">
      <strong><a ng-href="https://npmjs.com/package/\{\{random.name\}\}">\{\{random.name\}\}a>strong>
      \{\{random.description\}\}
    div>
    <hr>
    <table>
      <tr ng-repeat="package in packages">
        <td><a ng-href="https://npmjs.com/package/\{\{package.name\}\}">\{\{package.name\}\}a>td>
        <td>\{\{package.description\}\}td>
      tr>
    table>
  body>
html>

You can view 1 comment or e-mail me at sacha@sachachua.com.

Building a simple sketch navigator for myself

2019-06-12T18:58:05Z

In 2015, I built the habit of drawing daily index-card-sized thoughts. Some days, I reached (or blew past!) my target of five index cards a day. Other days, I backfilled my daily journal entries based on fuzzy memories and my time logs. In February 2015, I started using a yyyy-mm-dd naming convention so that I could easily refer to individual cards. For a while, I tried to be fairly disciplined about organizing sketches into outlines and building up chunks into blog posts. That fell by the wayside when I found it difficult to concentrate, but I kept drawing anyway. I saved the images to my hard drive, started tagging them with keywords in the filenames, and uploaded my sketches to Flickr as a way to back up and share my files.

I've recently been exploring ways to take advantage of the Samsung tablet that W- bought some time ago. It works wonderfully as a second screen that I can use to review an old sketch while I draw another one. The FlickFolio app lets me do a random slideshow of the past 2000 images in my photostream or in an album. This turns out to be an excellent way to jog my memory and prompt me to revisit questions or decisions.

2016-01-22d What do I want from my idea pipeline – index card #zettelkasten #ideas #thinking #questions #index-cards.png

Now those fragmented thoughts are starting to pay off. Many of my old index cards are on topics I haven't thought about in a while. It's fun to see what I've learned in the meantime, or to follow up on things I've forgotten. It's like I'd been filling an idea pipeline or seeding an idea garden. I get this steady stream of questions, decisions, observations, memories, ideas, and notes, and I can build on those prompts instead of having to start from scratch.

2016-01-18g How do I want my sketches to help me think – index card #drawing #sketches #zettelkasten #notes ref 2015-11-18a.png

I want these index cards to help me shuffle ideas and possibly see serendipitous combinations. They're small, so capturing thoughts on them doesn't require as much effort as, say, writing a blog post. They're a good way to smooth out and organize thoughts, and I can chunk up those thoughts into longer posts. The sketches are easy to review, too, so they work well as digital footprints. I re-read the year's blog posts as part of my annual review and sometimes I reread my posts on a monthly basis as well, but it's not as immediate or as wide-ranging as flipping through a random selection of images.

A random slideshow is a good start, but I've been meaning to make a sketch browser that's a little more tuned to what I want.

2016-01-18d What do I want in a sketch browser – index card #drawing #coding #plan.png

I wrote a simple NodeJS server that I can run on my laptop and access from the tablet (or other devices) while I'm on my home network. At first, I was trying to figure out an interface that would let me navigate by month/week/day, but then I realized that something simpler might be a more useful way to begin.

I started off by making it display random sketches:

Then I extended it to let me browse by tag:

And then to see a list of tags by frequency:

Or by alphabetic order, since that might be handier if I'm trying to look for something specific on a mobile device

I like distinguishing between daily/weekly/monthly/yearly reviews and non-journal sketches, too:

It's nice to be able to build custom little tools like that. =)

Here's the script on Github, in case you're curious or you want to build on the idea.

You can view 1 comment or e-mail me at sacha@sachachua.com.

Scripting and the grocery store flyer

2015-12-18T22:22:44Z

We plan the week’s meals around the grocery store flyers, taking advantage of what’s on sale (chicken, ground beef, etc.) and stocking up when the opportunity presents itself (for example, diced tomatoes or cream of mushroom soup).

The flyers are usually delivered on Thursdays. We do most of our grocery shopping at No Frills because it’s convenient and almost always a good price, but sometimes we’ll go to Freshco or Metro if there’s a particularly good sale. I might flip through the other flyers if we’re looking for something in particular, but most of the time, we just toss them out. I’d love to opt out of paper flyers, but that doesn’t seem to be an option in our neighbourhood.

It doesn’t take a lot of time to review the flyers, but I figured it would be fun to write a script that highlights specific items for us. Now I have a script that parses the output of the No Frills accessible flyer to create a table like this:

Y	Clementines	2.47		2 lb bag product of Spain $2.47
Y	Smithfield Bacon	3.97		500 g selected varieties $3.97
Y	Thomas’ Cinnamon Raisin Bread	2.5		675 g or Weston Kaisers 12’s selected varieties $5.00 or $2.50 ea.
Y	Unico Tomatoes	0.97		796 mL or Beans 540 mL selected varieties $0.97
	Fresh Boneless Skinless Chicken Breast	3.33	2.78	BIG Pack!™ DECEMBER 18TH – 24TH ONLY! $3.33 lb/$7.34/kg save $2.78/lb
	Purex	3.97	2.02	2.03 L $3.97 save $2.02
	Frozen Steelhead Trout Fillets	5.97	2.0	filets de truite $5.97 lb/$13.16/kg save $2.00/lb
	Heinz Tomato Juice	0.97	1.52	1.36 L selected varieties $0.97 save $1.52
	Nestlé Multi-Pack Chocolate or Bagged Chocolate	2.88	0.61	45-246 g selected varieties $2.88 save 61¢
	Source	4.97	0.5	16 x 100 g selected varieties $4.97 save 50 ¢
	Franco Gravy	0.67	0.32	284 mL selected varieties $0.67 save 32¢
	Ocean Spray Cranberry Sauce	1.67	0.32	348 mL whole or jellied $1.67 save 32¢
	10 lb Bag Yellow Potatoes, 5 lb Bag Carrots or 10 lb Bag Yellow Cooking Onions	1.87		product of Ontario, Canada no. 1 grade or 5 lb Bag Rutabaga product of Canada, no. 1 grade $1.87
	Betty Crocker Hamburger Helper	1.5		158-255 g or Mashed or Scallop Potatoes 141-215 g selected varieties $3.00 or $1.50 ea.
	Blackberries	1.25		6 oz product of U.S.A. or Mexico DECEMBER 18TH – 24TH ONLY! 4/$5.00 or $1.25 ea.

(more lines omitted)

The table is sorted by whether the item name matches one of the things we usually buy (first column: Y), then how much the sale is for, and then the name of the item. Over time, I’ll add more things to the priority list, and the script will get smarter and smarter.

I can use Org commands to move the rows up or down or remove the rows I’m not interested in. Then I can take the second column of the script’s output with Emacs’ copy-rectangle-as-kill command (C-x r M-w), and paste it into OurGroceries‘ import dialog. That builds a shopping list that’s sorted by the aisles I’ve previously set up, and this list is synchronized with our phones.

I’ve added the script to https://github.com/sachac/scripts/blob/master/check-grocery-flyer.js, so you can check there for updates. The flyer URL and the list of staples are defined in a separate configuration file that I haven’t included in the repository, but you can probably come up with your own if you want to adapt the idea. =)

Here’s the source, just in case:

#!/usr/bin/env node

/*
  Creates a prioritized list based on the flyers, like this:

Y Clementines 2.47    2 lb bag product of Spain $2.47
Y Smithfield Bacon  3.97    500 g selected varieties $3.97
Y Thomas' Cinnamon Raisin Bread 2.50    675 g or Weston Kaisers 12's selected varieties $5.00 or $2.50 ea.
Y Unico Tomatoes  0.97    796 mL or Beans 540 mL selected varieties $0.97
  Fresh Boneless Skinless Chicken Breast  3.33  2.78  BIG Pack!™ DECEMBER 18TH - 24TH ONLY! $3.33 lb/$7.34/kg save $2.78/lb
  Purex 3.97  2.02  2.03 L $3.97 save $2.02
  Frozen Steelhead Trout Fillets  5.97  2.00  filets de truite $5.97 lb/$13.16/kg save $2.00/lb
  Heinz Tomato Juice  0.97  1.52  1.36 L selected varieties $0.97 save $1.52
  Nestlé Multi-Pack Chocolate or Bagged Chocolate 2.88  0.61  45-246 g selected varieties $2.88 save 61¢
  ...
  */

var rp = require('request-promise');
var cheerio = require('cheerio');
var homeDir = require('home-dir');
var config = require(homeDir() + '/.secret');
var staples = config.grocery.staples; // array of lower-case text to match against flyer items
var flyerURL = config.grocery.flyerURL; // accessible URL

function parseValue(details) {
  var matches;
  var price;
  if ((matches = details.match(/\$([\.0-9]+)( | )+(ea|lb|\/kg)/i))) {
    price = matches[1];
  }
  else if ((matches = details.match(/\$([\.0-9]+)/i))) {
    price = matches[1];
  }
  else if ((matches = details.match(/([0-9]+) *¢/))) {
    price = parseInt(matches[1]) / 100.0;
  }
  return price;
}

function getFlyer(url) {
  return rp.get(url).then(function(response) {
    var $ = cheerio.load(response);
    var results = [];
    $('table[colspan="2"]').each(function() {
      var cells = $(this).find('td');
      // $0.67  or  2/$3.00 or $1.25ea
      var item = $(cells[0]).text().replace(/^[ \t\r\n]+|[ \t\r\n]+$/g, '');
      var details = $(cells[1]).text().replace(/([ \t\r\n\u00a0\u0000]| )+/g, ' ').replace(/^[ \t\r\n]+|[ \t\r\n]+$/g, '');
      var matches;
      var save = '';
      var price = parseValue(details);
      details = details.replace(/ \/ [^A-Z$]+/, ' ');
      if (details.match(/To Our Valued Customers/)) {
        details = details.replace(/To Our Valued Customers.*/, 'DELAYED');
      }
      if ((matches = details.match(/save .*/))) {
        save = parseValue(matches[0]);
      }
      results.push({item: item,
                    details: details,
                    price: price,
                    save: save});
    });
    return results;
  });
}

function prioritizeFlyer(data) {
  for (var i = 0; i < data.length; i++) {
    var name = data[i].item.toLowerCase();
    for (var j = 0; j < staples.length; j++) {
      if (name.match(staples[j])) {
        data[i].priority = true;
      }
    }
  }
  return data.sort(function(a, b) {
    if (a.priority && !b.priority) return -1;
    if (!a.priority && b.priority) return 1;
    if (a.save > b.save) return -1;
    if (a.save < b.save) return 1;
    if (a.item < b.item) return -1;
    if (a.item > b.item) return 1;
  });
}

function displayFlyerData(data) {
  for (var i = 0; i < data.length; i++) {
    var o = data[i];
    console.log((o.priority ? 'Y' : '') + '\t' + o.item + "\t" + o.price + "\t" + o.save + "\t" + o.details);
  }
}

getFlyer(flyerURL).then(prioritizeFlyer).then(displayFlyerData);

I’ll check next week to see if the accessible flyer URL changes each time, or if I can determine the correct publication ID by going to a stable URL. Anyway, this was fun to write!

You can e-mail me at sacha@sachachua.com.