Notes from GDS Tech Talks – Interoperability and Open Standards – 2020-02-03

These notes are just an attempt to capture some of the key points from the talks. I haven’t supplemented with anything personal so everything here should be considered my (potentially incorrect) interpretation of the speakers’ content.

Talks

“What’s new in open standards / open web” by Daniel Appelquist (@torgo), Samsung Internet & W3C

Daniel has kindly shared the link to his slides here: https://docs.google.com/presentation/d/1Hm7DMFLc8vgAMZUU52G9tXcxHbgSbTJvJIHTY8I1muo

Talk from Daniel Appelquist (@torgo). Dan does a number of things in open standards space; he works at Samsung Internet (android web browser with a strong focus on user privacy), and he is a member of the W3C Technical Architecture Group (TAG).

The TAG (https://www.w3.org/2001/tag/) ponders deep questions about the design of the web, publishes findings, and helps coordinate cross-technology architecture developments.

Some of the new things coming along are:

  • Connections between the web and your device e.g. WebNFC, Web bluetooth, Web USB, Serial API, Context API(?)
  • SMS Reeceive API, Contats API, Clipboard Access API, File System APIs…

New features that make the web able to develop a richer experience:

  • Service Worker, Manifest File, Push Notifications
  • Badging API (to enable an icon on desktop / home screen to show some richer information e.g a number that is programmatically set)
  • Web Assembley
  • WebGL
  • WebXR (bringing augmented / virtual reality use cases to the web)
  • Web payment

New layout capabilities:

  • CSS Grid
  • “Houdini” APIs (the next step in extending style, enabling you to script style within the Javascript layer. “Houdini” in quotes as it’s the name of the ‘project / team’ not necessarily the official name of the APIs)

New comms capabilities:

  • WebRTC, WebSockets, Streams, Web Transport

Enhancements to Web Security:

  • Feature policy, Signed Exchanges & Packaging

Progressive Web Apps
Intended to be inherently multi-device, multi-os, multi-browser. Grew out of a movement to make sure web apps on your phone are available offline, and to have more features that people expect from applications themselves.

What makes the web ‘open’?

  • BUilt on open standards
  • Based on user needs
  • Transparent and open processs
  • Fair access
  • Royalty Free
  • Compatible with open source
  • Multiple implementations

What makes a standard open?

  • Collaborating between all interested partiens, not just individual suppliers -transparent and published decision-making process that is review by subject matter experts

Wide Review

  • The open tandarsds proces itself – call for reviews, public comemnt period….
  • Github issues in the repo where the standard is being work on
  • Discussions in browser-specific or engine-specific mailing lists (e.g. Chromium)
  • Review by vertical groups in W3C (Accessibility, Internationalisation, Privacy)
  • Review in standards groups outside W3C
  • TAG review
  • People talking on the internet

Ethical Web
How do we build things that are not only subject to strong technical and business review, but are also ethical?

Rules that we se for ourselves, encoding certain lmoral values into rules that guide conduct. e.g. The Hippocratic Oath (“Do no harm”).

Lots of examples of Tech Ethics out there:

ACM Ethics

Google AI Ethics

Mozilla Manifesto

  • should enable civil discourse, human dignity, individual expression

“The web already encodes ethics.”
e.g. Accessibility (A11Y)

  • WCAG
  • ARIA
  • Silver
    UPcoming – AOM)

Internationalization (I18N)

  • Make it possibe to use Web Tech with different languages

Privacy & Security

  • A workshop called out that “Pervasive Monitoring is a Threat to the Web” – resulted in a call to move the web to HTTPS
  • W3C Privacy & Security Questionnaire – could we adopt this for health?

Last year, Dan and Hadley Beeman worked on W3C TAG Ethical Web Principles.

Some bits from that:

“The Web Should Not Cause Harm to Society.”

There are lots of examples of this already:
– Filter bubbles
– Clik-bait
– Fake news
– Addictive behaviours

The web must support health community and debate.

Consider:

  • Harassment
  • Algorithms that prioritise “engagement”
  • Content warnings

“The web should prioritise the needs of marginalised over the comfort of the priviledged”

The web is for all people.

The web must enable freedom of expression…
– …as long as it doesn’t contravene human rights
– Passive monitoring and surveillance can have a chilling effect
– Freedom of expression does not mean free speech everywhere
– Hate speech, harassment, or abuse may reasonably be denied a platform.

The web must make it possible for peple to verify the information teh see: e.g.

  • Credible web activity
  • Extensions that verify information or verify sources (Dan uses “News Guard”
  • Data journalism

Web must enhance individuals’ control and power:
Consider harmful use cases such as:

  • Stalkerware
  • IoT used by abusive partners

The web must be an environmentally sustainable platform:
Consider:

The web is transparent:

  • “View source”
  • Transparency enables:
    • learning
    • research & audit
    • extensions

The web is multi-browser, multi-os, and multi-devices:

Multiple implementations help to keep the web open
People should be able to render web content as they want.

How do we apply these principles in real life?

  1. Start with web ( “A responsive website is usually the best way to build a service that will work on whatever device or browser they choose to use.
  2. Respect peoples’ privacy
    • Do you really need that info?
    • Consider GDPR
    • Don’t collect gender information unless you really need it.
  3. Use permissions-requiring features, & especially notifications, sparingly.
  4. Use the URL
  5. Test in multiple browsers
  6. Design for marginalised groups and communities
    • Tools for reporting harassment.
    • Features that discourage negative behaviours
    • Features that encourage good behaviours:
      • Content warnings
      • Text descriptions
  7. Respect humans and their choices
    • Let people delete their accounts
    • Dont fingerprint people (or their browsers)
  8. Get involved in web standards
  9. Join a discussion on a new technology / standard proposal

“Intro to OpenAPI” by Lorna Mitchel (@lornajane), Nexmo

APIs are the enginer of modern software development. API descriptions power up our API workflows.

Spec-first API Design

Who writes API specs?

<some missed notes>

Having tech writers on the team makes a massive difference. Tech writers have to be incredibly sharp and concise.

Should I write API descriptions for my existing APIs?

Yes – retrofitting can be painful, but helps highlight places where you’re making mistakes you shouldn’t be in your existing APIs.

API Description Languages

There are some proprietary API description languages e.g. API Blueprint (from Apiary) and RAML from Mulesoft.

OpenAPI (https://swagger.io/specification/) is an open standard (formerly known as “Swagger”).

<Lorna did a walkthrough of an OpenAPI specification example which I didn’t attempt to note>

API descriptions are not just important for understanding your APIs, they are going to enable your users to discover your APIs.

OpenAPI allows you to define components once, and refer back to them from multiple endpoints within the API specification.

How do you edit OpenAPI specs

They are just text-based specs, so you can use your text editor. Lorna uses Atom with a plugin (possibly still called “Swagger”).

Stoplight Studio is a free-to-use IDE for working with OpenAPI specs. There is a desktop and web-based version. https://stoplight.io/p/studio

Spectral Linting tool lets you lint your OpenAPI specs using pre-defined rules https://github.com/stoplightio/spectral

What can you do now?

Generate Documentation – use your OpenAPI spec as the source for your API reference documentation

  • There are a choice of tools you can use to do this
  • The content and presentation of your documentation is completely separate
  • The OpenAPI spec has support for examples which is great for enhancing your documentation

Redoc is a documentation-generator that builds from OpenAPI specs https://github.com/Redocly/redoc

Postman (https://www.getpostman.com/) lets you import the OpenAPI spec into a Collection, with requests already set up for you to easily create requests to support debugging etc.

Prism lets you turn an OpenAPI spec into an API server https://github.com/stoplightio/prism

(The OpenAPI spec has recently had Webhooks support added – will be in the 3.1 release. Lorna expects Prism to support this pretty quickly.)

Code Generation – OpenAPI Generator https://github.com/OpenAPITools/openapi-generator – lets you take an OpenAPI specification and generate lightweight API wrapper client code in your chosen language. It’s a quick way to create sample client code for your APIs.

Finding OpenAPI Tools

https://openapi.tools – a community listing of OpenAPI tools – you can submit contributions and additions.

What lessons has OpenAPI learned from WSDL and SOAP?

  • The ability to quickstart developers – SOAP provides the WSDL that developers could just feed into their dev stack and automate lots of the work. OpenAPI is building on this concept, but adding lots of new stuff.

Why haven’t you mentioned Swagger?

Swagger is more than one thing. It is a tradename for the toolset from which the OpenAPI was specification emerged. Formerly, the specification standard was called ‘Swagger’ – this was then renamed to OpenAPI 2.0.

Swagger (https://swagger.io/) still provide tools for working with APIs, and their tools support the OpenAPI specification.

Does OpenAPI handle GraphQL?

Not really – OpenAPI is really REST-ish. If you’re working with GraphQL, then you’re best off using the tools that are provided for that.

OpenAPI isn’t necessarily aiming to converge with GraphQL world, as it’s not a great fit in terms of paradigm.

Does OpenAPI make recommendations on where the descriptions should be hosted?

There are no official recommendations on where you should host the descriptions (as in which path to host the documentation on). Lorna feels like OpenAPI probably wouldn’t recommend hosting your descriptions on the API paths themselves.


“API First for the humans and the machines” by Kin Lane (@kinlane), Postman

Going to talk a bit more about the governance around API development.

It’s important to set the stage when talking about APIs – Kin focuses on web APIs (as opposed to browser APIs, hardware APIs etc.)

The latest wave of web APIs was set into motion by eCommerce giants like Amazon, Ebay etc.

Following that, startups began seeing APIs being more than just about ecommerce, but about social interactions.

In 2006, a big step was when Amazon realised you could deliver infrastructure using APIs.

Why are we doing web APIs?

Originally, the commerce giants had data that they needed to make available to people on the web. APIs were being used to publish and syndicate content to websites. APIs allowed you to publish the data to websites, mobile applications directly fro the data source.

We also saw the creation of widgets, allowing people to take that data and make it available embedded within other websites.

Data needed sharing to trusted partners, e.g. sharing water utility data across the US.

With mobile phones taking hold, people realised APIs could be used to drive sensors, and other data sources connecting everyday items to the internet.

API Development Lifecycles

An API lifecycle can take an average of 3-12 months to deliver each version of an API, using a code-first approach. It can take that long to put something out there, get feedback from the people who will use it, and then iterate on it.

The introduction of tools like Apiary (Kin largely credits them with being first) allowed people to design, refine, document, and iterate their API designs without writing any code. Processes that would take years could take months or even days.

Postman is an API platform, with the majority of usage being using it as an HTTP client. it has evolved along the way, with much more functionality being added to it. Today you can use it to actually design your API.

<Kin gave a walkthrough of designing an API using Postman>

Demonstrated creating a new API Path in Postman, and then added a JSON example to the request. You can then create a Mock API, which gives you a published Mock API URL, which returns the example JSON response you added. The response could be something other than JSON: it could be YAML, or CSV etc.

As Lorna demonstrated with Stoplight, Postman can create documentation for sharing with users. As Lorna also demonstrated, you can import OpenAPI specifications into Postman. A new feature also lets you add two-way sync with the git repository containing your OpenAPI specification.

This integration gives a lot of flexibility around how users can work with your APIs. You can either build entirely within Postman, or plug into an existing API dev lifecycle – potentially allowing different members of your team to use different tools e.g. you could be using Postman, whilst others are using Stoplight tools.

Letting developers build against your mock APIs

Having Mock APIs let’s people start to design the things that are going to interact with those APIs. e.g. mocking your mobile apps, your display widgets

The advantage of getting people doing this very quickly is that you start getting feedback on the API design, and you can feed this back into your API design really quickly.

It allows your API users to help define the API tests that are used to validate each API response. Having users involved in the design, allows you to create API contracts by applying API tests that everyone contributes to.

Deploying your API

Once your API is ready it can be deployed using a gateway or other approach. There are some services which can take an OpenAPI specification and set up your management layer – although almost all require you to wire up the endpoints to the data layer behind.

API Management

  • All APIs should have a management layer.
  • All should have security, rate limits, analytics. API management is what will protect you if there are issues. There are many API management options out there now.

Documentation

All APIs should be clearly documented.

Monitoring

  • Monitor from different regions around the world.
    • Your monitors can have issues, you need to build in monitoring resilience.

Testing

  • Make sure API tests are avaialble to be run manually against the API.
  • API tests should also be available to be run as part of a CI/CD pipeline (automated tests)

Security

Ensure security – You should use OWASP, and associated tools, to make sure you are covering common security guidelines and avoiding common security vulnerabilites.

Support

You need to make sure that there is a strategy for APIs to have owners and support avaialble.

Communications

Every API should be announced. There should internal and external communications around every API release. All developers should have to step up and be part of that communications strategy. They need to be able to communicate the value and detail of the API releases.

Observability

“Leveraging all of the outputs from API operations to ensure all APIs are observable and seen by stakeholders.”

Governance

Establishing governance of design, mocks, docs, tests, and all other parts of the API life cycle.

How do you define what API design should look like?

Make all teams move to an API-first strategy. It forces team to be having conversations early on.

You should have a documented approach to how you govern your APIs.

“Governance wraps up the full life cycle, and allows teams to understand how they should be building and operating APIs.”

Enterprise

Mainstream companies are getting more organised about how they deliver APIs, establishing common approaches to delivering APIs across their teams.

Working with government

APIs can be an answer to departments hoarding data for ‘power’. They also affect the politics though. For example, gov departments might be paid to hold the ‘authoritative list’ of something, so unsurprisingly there are multiple ‘authoritative lists’.

APIs can tackle the data silos, and enable better cooperation between gov departments.

API Advocacy

The most important tools we have are evangelism and advocacy for API-first approaches. It’s important to tell stories of what’s going on, and to help developers (and others) understands the human effects and benefits of APIs. It’s important to be able to articulate the human aspect of APIs – not just the technical. We need the conversations to focus on the real-life stories.

Storytelling

Continued storytelling is important to keep API approaches alive, to educate new champions and advocates, to keep up the momentum. Stories are all that matter – communicating the impact we make on our environment, our children, our veterans etc.