Digital Access Made Easy Steve Lee's OpenDirective blog

Sending emails with the G Suite Gmail API

If you have a G Suite instance then sending emails from a backend using Gmail turns out to be really simple. Using the Gmail API rather than SMTP and enabling G Suite global delegation means you can simply send email as a G Suite user. Here's how I did it with Netlify Functions.

Email options for backend

In my previous post on the Tools with a Mission (TWAM) app I mentioned the need to send email notifications of various events. TWAM is a non profit usr of Google’s G Suite and all users requiring notifications have appropriate G Suite Gmail addresses. The app is using Google Sheets as the backend database and it makes sense to also use Gmail to send emails, rather than using yet another service.

It might seem obvious to just use the same SMTP API that email clients use to send and receive via gmail. However, a little research indicates this approach has issues, largely as Google are aggressive in disabling accounts which seem to be suspect, with little recourse to restoration. We obviously can’t risk that for a mission critical app.

As the TWAM app has a backend of Netlify Functions (which simplify AWS Lambda) running on nodejs, another sensible choice is using nodemailer which has a facade for Gmail. Again however, the documentation indicates problems with Gmail and it’s a heck of a lot of abstraction on top of the the Gmail APIs, with support for many other transports.

Thus direct Gmail use is my preferred solution, assuming it can be made to work with a reasonable “pain threshold”. As usual with Google APIs, the documentation is poor, confusing and often downright misleading due to multiple versions being around on the interwebs. Worse, when trying to find solutions in Stack overflow etc, well, “there be Dragons” as they used to say on old maps for unknown dangers.

But in the end after many, many, failed and very frustrating attempts the solution using the Gmail REST API directly is pleasantly simple. You just need to get all the parts lined up “just so”. Google provide a JavaScript API for node , but that is a monster monolith covering ALL Google’s APIs. I don’t expect so much deployed code will run up big costs on Netlify Serverless Functions, but the thought of using such a “Swiss Army Knife” binding leaves a nasty taste in my mouth. REST will do just fine, thank you; after all, it’s the way of the web.

Code

Reading up on the subject of using the Gmail API throws up several complexities that it turns out are just not required:

No need to get the user to authorise using an OAuth/OpenID flow

This is just as well because we want to send emails as the app backend, not a specific user who is logged into the app. To do so, we can set up a Google Service. This actually uses OAuth JWTs “under the bonnet” but it’s much simpler for us to use than the usual flows. The service use an email address for identification and a private key for security. We can use the individual google-auth-library package that is part of the Google JavaScript APIs. The final piece of the puzzle is to use a G Suite ‘Global Delegation’ setting to allow the Service to send emails as a user. This way, the app calls the REST API and Gmail sends the a email as the specified Gmail user. To set this up See the Google documentation and ensure the correct scopes are enabled for sending emails (the used value of ‘https://mail.google.com/’ seems too course and can probably be refined).

No need to BASE64 encode content

While nearly all examples show the HTTP message body for the email being BASE64 encoded this appears uneccessary. You only need to specify the correct Content-Type header. Anyway, BASE64 encoding is certainly not part of the RFC 822 email specification. Perhaps we’ll hit some content that requires it but for now I’m only sending single part text content (and unicode works just fine).

No need to use NodeFetch or NAXIOS

google-auth-library includes a dependency on Gaxios, a version of AXIOS that works just fine for our purposes. It’s called via a wrapper that retries under some auth error circumstances.

The email functions

sendRawEmail calls the Gmail REST endpoint with appropriate security provided via Netlify variables in deployment or a .env shell environment configuration file during development.

sendEmail builds up a RFC 822 message body for common email fields and calls sendRawEmail.

// file - functions/_gmail.js

if (!process.env.NETLIFY) {y
  // use .enc file for local dev and assume netlify variables in CI
  require('dotenv').config()
}

if (!process.env.GOOGLE_SERVICE_ACCOUNT_EMAIL)
  throw new Error('no GOOGLE_SERVICE_ACCOUNT_EMAIL env var set')
if (!process.env.GOOGLE_PRIVATE_KEY)
  throw new Error('no GOOGLE_PRIVATE_KEY env var set')
if (!process.env.GMAIL_SENDING_USER)
  throw new Error('no GMAIL_SENDING_USER env var set')

const { JWT } = require('google-auth-library')

// For this to work you must create a service and enable Domain wide delegation for the service
// https://developers.google.com/admin-sdk/directory/v1/guides/delegation
// Set env vars for the service key in GOOGLE_SERVICE_ACCOUNT_EMAIL & GOOGLE_PRIVATE_KEY
// GMAIL_SENDING_USER is the email address that the service delegates for
// ensure you set the used scopes here when enabling the global delegation
async function initServiceClient() {
  return new JWT({
    email: process.env.GOOGLE_SERVICE_ACCOUNT_EMAIL,
    key: process.env.GOOGLE_PRIVATE_KEY.replace(/\\n/g, '\n'),
    scopes: ['https://mail.google.com/'], // better to pick narrower scopes
    subject: process.env.GMAIL_SENDING_USER,
  })
}

// email body is to rfc822. (From: is ignored and GMAIL_SENDING_USER used ) eg
// To: someone_else@example.com
// Subject: An RFC 822 formatted message
//
// This is the plain text body of the message. Note the blank line
// between the header information and the body of the message.
exports.sendRawEmail = async function (emailBody) {
  const client = await initServiceClient()

  // see https://github.com/googleapis/gaxios
  const options = {
    method: 'POST',
    url: `https://gmail.googleapis.com/upload/gmail/v1/users/me/messages/send`,
    headers: {
      'Content-Type': 'message/rfc822',
    },
    body: emailBody,
  }
  return await client.request(options)
}

const field = (f, v) => (v ? f + ': ' + v + '\r\n' : '')

exports.sendEmail = async function ({
  to = undefined,
  cc = undefined,
  bcc = undefined,
  subject = '',
  message = '',
} = {}) {
  if (!to) {
    throw new Error('"To" field is required')
  }

  const email = `${field('Subject', subject)}${field('To', to)}${field(
    'cc',
    cc,
  )}${field('bcc', bcc)}\r\n${message}`
  return exports.sendRawEmail(email)
}

Extra - templates in 3 lines

I also required a simple template for the emails. Something like mustache (or handlebars) is just fine. Lodash provides the necessary.

const template = require('lodash.template')

function parseTemplate(templ, data) {
  const options = { interpolate: //g } // mustache style
  const compiled = template(templ, options)
  return compiled(data)
}

 const notification = {
      to: recipient.email,
      subject: parseTemplate(email.subject, templData),
      message: parseTemplate(email.message, templData),
    }
  sendEmail(notification)

TWAM Proof of Concept app - using Netlify, Eleventy and Google Sheets

A new project with yet a different stack. This time it's a Jamstack static site with a serverless back end using most of Netlify's services. On this platform Eleventy provides templated page build with no front end framework in sight. Google Sheets is used as a database. This post explains some of the technical decisions made and challenges met.

Digitising Tools with a Mission’s process

Tools with a Mission (TWAM) recycle and refurbish tools, sewing machines and computers. It then sends them to the developing world for livelihood creation. Their processes are reasonably complex and currently use a mixture of Google Apps tools, including GMail and Sheets. I’m working in a small team to help move more of the process to an integrated ‘web app’ experience. This will support customer applications for tools, evaluation, approval and other steps through to shipping in containers.

This is a pretty standard setup of an app with forms plus a back office process which defines the workflow. Thus we looked at the usual candidates for forms and workflow. I personally find such systems to be a quite restrictive, especially if they only use a GUI and cannot be data or code driven. Given TWAM’s embedded use of Goggle Apps we also considered Google Action script which is basically a hosted JavaScript environment with some support for creating web apps and APIs.

In the end however, my very positive experience with using Netlify Deployments along with Eleventy for https://musicpracticetools.net meant I was keen to try more of the Netlify portfolio of serverless services. However, for the database, rather than the pNetlify partner noSQL Fauna, I decided to try using Google Sheets. Using sheets opens up the possibility of easy access to the data and ad-hoc processing using office software skills rather than full developer skills. In addition, Gmail is used for inter-person messaging and for automatic notifications. Again, why code a new messaging solution when a good one exists and fits in with people’s work patterns. So, the stack chosen for the proof of concept is:

  • Eleventy with nunjucks templates
  • Good ol’ HTML, CSS and Javascript - ie no framework
  • Google Sheets API for database
  • Gmail API for notifications
  • Netlify deploys, forms, functions, identity and role based gating
  • Netlify dev - not really part of the stack but excellent

This stack supports a representative slice of functionality including user management, form filling with editing and notifications. The proof of content flow is that someone makes an application and can check back later to see if successful Someone else reviews a subset of applications filtered by country and adds evaluation notes and assigns an accept/reject status.

Tricky areas

I’d already ironed out any small issues with Eleventy and nunjucks in my last project so just used the same config again. Netlify deploys are really smooth to use with github integration and pretty much just work; as do the forms which invoke a function when submitted and also functions which wrap AWS Lamda, making them a “doddle” to use.

The fun and games came with Google Sheets and Netlify identity used in conjunction with the role based gating.

Google Sheets

Google provide a REST API for Sheets and also the Action Script environment with support for creating HTTP APIs. They also maintain a npm package binding JavaScript to ALL their APIs, though there is a simpler independent package just for sheets. We can most easily call the JavaScript binding from our Netlify Functions rather than poking the REST API via fetch. I decided to use the simpler google-spreadsheet package, though it turns out I might switch to the Google monster package as it supports batching API calls for efficiency.

But we soon hit an major issue. For some reason the latest v4 of the Google Sheets API does not support any filtering when fetching sheet cells. And v3 is about to be dropped. That’s obviously a scalability problem! We don’t want to load all columns and rows into memory for processing. I looked at using Action Script to create an API which does work but there are two issues here. 1) the script have to be edited in Google’s in browser editor which is a terrible experience compared to something like VS Code. Perhaps more serious is the fact that in order to expose the API endpoint the Google Apps setting to allow anyone to share items publicly must be enabled. That’s a potential security issue and might not be welcome.

After much head scratching I found a work around to this. YOu can use a sheet formula to process the sheet data as required and return the filtered row set. Further more, I decided to reduce coupling and error possibilities, that rather than keeping the formula in the spreadsheet we’d push it from the app and read the results back. In addition, in order to have a have chance of handling multiple accesses at once a new sheet is created for the formula and then deleted. So the process to get a filtered row set is

  1. Create a new worksheet in the spreadsheet
  2. Push the Formula to a cell so it runs
  3. Fetch the results
  4. Delete the worksheet

This works well but does seem a little “hacky” and I still need to test with multiple simultaneous updates. It is also slow, taking around 4s. This appears to be purely communication time with so many round trips. Adding a thousand rows to the sheet makes little difference to the timing. The official Google npm package provides batching mode which should speed things up. You need some good sheet formula skills but can develop in the sheet and move to JavaScript once it all works. In order to further reduce coupling and hardcoded values the formula returns the size of the result set and column names as well as the data itself to the app. The app can read that data first and then use it to fetch the correct cells.

Netlify Identity and Role Based Gating.

Identity along with account management is always a complex topic. Fortunately, Netlify provide a register / login widget that is perfectly serviceable, though nowhere near as flexible or polished as the one from Auth0. I found I could improve the UX to my liking with a little script, though have not yet succeeded with the CSS tweaks I want. Identity is based on stateless JWT tokens as you’d pretty much expect these days. These are flexible and you can pass the user’s id_token to your Functions to provide access control and user info. This way, Netlify manages the users and their details are passed around the front and back ends in the tokens.

The feature that I really appreciate is Role Based gating. This means access to specific app page URIs is controlled declaratively right on the CDN edge almost without any code required. When a user registers and logs in a Function is called and this can return the users roles. Then a config file is used to define which roles are required per URI. And it just works.

Well almost. I hit a lot of strange behavours and errors with access, including an apparent failure when logging in on another browser when still logged in. It turned out to be down to my limited cognitive model of how the pieces work together, not helped by very limited documentation. The gating and identity work differently but do get synchronised if you are careful. This is where Netlify’s Community forum came to the rescue in the form of one Jon Sullivan. As a volunteer Jon is incredibly knowledgeable and definitely goes the extra mile, and then some. After a long an fascinating discussion covering how authentication and gating interact and how to handle refresh tokens I now have code which is reliable. It just took a very long time to get there. But we did get a 100+ post thread going, FTW.

Summary

A important point I should make clear is that ALL this functionality from Netlify is available on the free plan! Though, Forms do have their own pricing model after 100 submissions. That’s very impressive indeed and perfect for a charity with very little cash to spare.

The stack threw me a few big challenges but now it is working well and I think will scale well to the final app.

The code is available at https://github.com/OpenDirective/TWAM-App

Music Practice Tools

As an adult learner of bass guitar I find there's much to get to grips with. In addition to instrument technique there's a raft of musicality skills to work on. As a confirmed geek I've long thought about various neat musical projects using the web platform which is now very musically capable thanks to the Web Audio API. And then COVID19 hit us and my lessons became remote, requiring distant sharing of information with my teacher. Add to that my aspirations of keeping a musical journal and musicpracticetools.net was born as a hybrid blog/journal/app type thing.

Tools for practice and remote lessons

Fortunately, my bass teacher encourages me to develop all-round skills and a comprehensive daily practice has evolved. I have a task list to work through and while I could do so with a paper checklist, given my handwriting, I liked the idea of something interactive. Plus it could integrate with a timer so each task could be timed with an indication of when to move on to the next.

Every week we have a video call and holding my scruffy transcriptions up to the camera turned out a pain. It’d be much better if I could share nicely formatted “dots” via a web url. And what if the dots could be played so I could check the transcriptions as I made them?

Then there’s the minor annoyance of making repeated changes to the metronome bpm and picking random keys or exercise numbers.

My existing Youtube current time widget will be really useful when practicing music from that source. And a series of links to the music sections would provide a cool way to study song structure.

Other gadgets like a tuner seem unnecessary, though perfectly possible and would make interesting projects. however the solutions I have already are fine. That’s solutions like, er, a tuner. Though a single A 440 note would be a useful addition.

Finally, there’s that idea of keeping a journal, with notes, ideas, thoughts, a collections of tunes I’ve work on or would like to learn, and more.

But what is it exactly

So while this could be some form of web app I had something a bit different in mind. Basically a blog but enriched with on-page widgets to provide interactive tools.

Unlike web apps which are interactive with settings to adjust features, a blog has a definite “write and publish” life cycle. Interactive elements would be configured at write time but any changes will probably still need to be persisted. That might become a problem but when it’s just for myself that’s really not an issue. I’m happy to hack templated markdown in an editor (VS code these days) and deploy to publish. An app does offer the lucrative advantage of an easy route to monetization via subscripted SaaS, but eventually I decided to make it open source in case others find it useful during these difficult times. The route to using is a little more complex though than an web app, though a simple “deploy to XXX” button on git hub would be ok for technical users, as will hacking on templated markdown.

The Technical Stack

The metronome key requirement is it has to be easy to create and publish pages and also to include or develop new widgets. That means some sort of publishing framework that also supports client side components, ideal with custom templates rather than partial includes as they are a little prettier. While WordPress is an established solution I’d much prefer static hosting to n-tier dynamic architectures, for quite a few reasons. And these days here’s plenty of static site generators to choose from and even the static generation options for component frameworks like Svelte, Next and Nuxt. In the end 11ty won me over with it’s light weight simplicity and flexibility. While it works with many template languages for creating pages, Mozilla’s nunjucks offers most functionality in 11ty.

For client side we need to create widgets that enhance the static content. I’m happy as a “pig in muck” to craft good ol’ DOM manipulation code myself but it does tend to quickly get tedious and “boiler-platey”. So something is called for that helps without imposing the full overkill weight and fixed abstractions of a runtime framework like [insert your favourite here]. I’m was keen to use svelte as I think it gets so much right, including minimal load and runtime impact. However, it is usually used for building complete apps with a single component tree and that’s not what is wanted here. Svelte will build web components and that certainly a possibility, though at the expense of complicating the build process. There are certainly other interesting lightweight client runtime-only possibilities including Hyperapp and lit-element but in the end I stumbled upon alpinejs via Tailwind UI and it really hit’s the sweet spot for this application.

Now what about the creating the musical widgets using the web platform? Many moons ago when I was involved in Mozilla I was mightily impressed by the work at Seneca College led by David Humpreys. That work included a new audio API they were working on. Eventually that led the incredible Web Audio API that we now have to play with. However this is no easy-peasy high level musical tool like Microsoft DirectPlay (now depreciated). The Web Audio API requires a lot of bit twiddling so it’s wonderful to have the excellent ToneJS library to make it much easier to grapple with. For managing abstract musical concepts like scales we have tonal.

And then there’s abcjs based on the semi-standard musical score text language abc. This adds the icing on the cake providing SVG “dots” that can also be played. The only wrinkle is there is no WYSIWYG editor but rather you describe the notes in the abc language. But that’s just fine with me and works well in a publishing workflow.

Finally I had to figure persistence of widget state. That should be minimal but things like timer time need to work as you switch pages (ie across page loads in jamstack). We dont’ have cookies or invisible field type hacks as there’s no server. This isn’t a SPA keeping the top of the DOM tree in place between psuedo page updates. We could similar approach like turbolinks which keeps the Window object and swaps the document body for each page load. In the end I decided to use browser local storage. This requires careful lifecycle management to avoid problems due to lack of a single source of truth. We’ll also need to manual manage ids to void problems with page changes and may even need a form of garbage collection. Oh and I just figured there will be problems with multiple tabs. Plus the user will need reset options (or else we tell them to use the browser F12 tools). It’s not so surprising this is not a common approach. I may move on fairly soon.

Making it

Dev and build is handled by Nodejs (what else these days) but only for server side 11ty. The client side is pure un-built un-bundled un-munged HTML, CSS and JavaScript (like your grandma used to make), using global scripts. And why not? It works just fine and I’ll bundle etc if and when I need to. Code spliting is easy if required to reduce Javascript load time. I did look at browser modules but there’s a few quirks to solve (like modules are deferred which messes up code in the main html file)

And what about deployment? Well, while GitHub pages can be bent to work without Jekyll, they have limitations. GitHub now has Actions to provide Continuous Deployment (CD) etc. and there’s Zeit (I mean Vercel). But Netlify really seem to call me with their good range of services, free pricing tier and community vibe. They also provide Serverless Functions for if/when we do need backend functionality. Netlify have a headless CMS using GitHub for storage that would allow for an alternative in-browser editing / admin UI if that is ever required.

Outro

So there we have it. A perfectly serviceable stack for a slightly left-field blog/journal/app thing. So far it has proven more than adequate. I create widgets using client side Javascript, sometimes with Alpine, and 11ty shortcodes to make the markdown page editing experience quick and clean. Local dev of shortcodes with automatic browser update required a little trick with nodemon and browsersync but it works really well.

Here’s an example daily practice page as created and a tune page.

We’re really lucky to have so much great open source software to build on. Hopefully this adds a little useful something to commons.

Right, now I can concentrate much more on developing my bass chops!

Oh, wait, I just thought of a really cool new feature to add…..

The Open Communication Symbols Ecosystem

Many people without speech use symbols each and everyday to aid or enable communication with others. The current COVID pandemic means more people now find themselves literally speechless while on breathing support such as CPAP or are simply too weak to speak. Fortunately there is a growing number of free or open symbols sets and apps providing low cost tools.

AAC

Augmentative and Alternative Communication (AAC) is a term that describes a set of techniques and tools that support people who have difficulties communicating, especially if they do not have speech. Such difficulties may result from congenital disabilities such as Cerebral Palsey or may be acquired, for example Aphasia caused by a stroke.

The basic idea of AAC is to create printed or interactive electronic grids of symbols that capture common needs or thoughts that a person wants to communicate. The symbols may be use individually or strung into sentences by the user either pointing to them with a finger or by looking at them (eye pointing). Low tech versions are printed on paper and high tech solutions use ‘eye tracking’ and speech synthesis.

COVID19 has resulted in more people finding themselves unable to easily express their wishes or communicate with loved ones. While there are several proprietary AAC tools to support them there is also a growing number of free and open solutions. This is important as the experts in distilling the entire language needs of someone into a usable set of charts of symbols, Speech and Language Therapists (“SALT”s in the UK), are more often than not running on a “shoestring” budget (like most of our Health and Social Care sectors).

Mulberry Symbols

Some time ago I took over stewardship of the Mulberry Symbols which in addition to providing symbols aimed at adult users which are often unavailable in proprietary sets, are also low cost and freely sharable or modifiable. They are thus also compatible with flexible online use.

One of key questions in my mind is how can we make sure that SALTs can easily choose and use the symbols they want for users. While we provide a downloadable zip archive of all the symbol files they can be fiddly to create boards with. We also have a slightly “hacky” way to download individual symbols.

Print boards are usually made using a word processor or presentation software and the symbols then have to be inserted or dragged into place and then scaled. Use in AAC apps often involves other complexities, including the need to transform the symbols into a format that the app will accept. The Symbols also come with English names which makes it difficult to find and use in other languages. We do have a file mapping to Finnish names.

While we could expand the options for accessing and using Mulberry Symbols there are fortunately a growing ecosystem of open solutions that help address these issues.

The Open Symbols ecosystem

Global Symbols - Global Symbols collects and creates free and open AAC symbol sets, tools and resources. The Linguistically and culturally localised resources provide world wide access to appropriate pictographic based communication. Several of the resources described below are provided by the Global Symbols team and friends.

Open AAC - The Open AAC initiative exists to create and encourage open-licensed resources and tools to support AAC users. It introduces a number of key concepts including the Open Board Format discussed below.

Finding Symbols - Both Global Symbols and Open Symbols provide online tools for exploring, selecting and downloading symbols from a range of open sets, including Mulberry. Some symbols are also available in multiple languages They also offer other facilities for editing symbols or providing associated meta data.

Internationalisation and Localisation - This gets a little complex. Some symbol sets have symbols names in a specific language and the symbols might be culturally specific. A good example is the Tawasol Arabic set. If a symbol set is to be more globally useful then the names need to be available in multiple languages and some of the symbols will need alternatives. In fact some symbols may only be required for some cultures. And remember, culture and language are often not the same thing. Practically the symbols should be searchable across range of languages and cultures.

Open Source projects traditionally have very good internationalisation support and many community contributions take the form of translations. I hope the open symbols ecosystem will benefit from this crowd sourcing. In fact, Global Symbols has a translation tool that uses online translation tools to get an initial rough automated translation which can then be reviewed manually for correctness.

Creating Boards - selecting symbols and placing them into boards is a common requirement. When the board is to be printed Office documents are often used and templates and samples like those from the CommuniKate project can be really helpful. AAC apps usually have their own way of designing boards but now the Open Board Format (OBF) initiative is proposing an open and standard board format. This will allow easy interchange of boards between Apps and so open up the ecosystem for collaboration and innovation. I hope OBF will be adopted by all AAC solutions.

One App that makes the most of OBF is BoardBuilder from Global Symbols which allows you to create and modify boards, print them or export in the OBF format for use in AAC apps. A number of sample boards are also provided.

Creating, modifying and Sharing Symbols - Symbols sets are never complete and the users probably want individualised symbols. After all, symbols are often a person’s main or only language so they want them personal. Open Symbols provide an opportunity for community collaboration on symbols through the flexible permissive licenses that allow free modification, possibly with the condition to share new symbols under the same licence. It’s generally also good manners to provide new derived symbols back to the original project so others can benefit. Global symbols also provide a little online symbol creator tool.

AAC Apps - Both Cboard and Coughdrop are full featured Open Source AAC Apps. They support OBF and ‘books’ of connected boards where a symbol in one board links to another board, allowing for complex symbol collections. both provide speech output of sentences built with symbols. AsTeRICS is a flexible platform for AAC and alternative input; it now supports OBF and has a web version called AsTeRICS Grid.

Training - Creating usable boards is a skill requiring understanding of language and more. Fortunately, Global Symbols have created a number of free training resources in collaboration with UNICEF.

Come Join Us

So in summary, there is a growing ecosystem of open symbols and tools. Not only do these provide low cost solutions but they also enable collaborative community development so we can better support AAC users. This will work best when we have collaborators involved with all aspects of creating and using symbols. Please join us! Here’s the Open AAC slack channel.

Open Communication with Symbols

This content has moved to The Open Communication Symbols Ecosystem.