UK Public Sector Code by Organisation

A cookiecutter template for data science projects within His Majesty's Government and wider public sector.

Guidance for quality assurance of code for civil service researchers and analysts.

R package: generate best-practice stats spreadsheets for publication

Good Practice Tables - an XlsxWriter wrapper to write consistently formatted statistical tables to Excel.

ONS Design System

Development website for collecting and disseminating UK data for the Sustainable Development Goal global indicators.

Reusable map components for Svelte projects.

afcharts is an R package for creating accessible plots by the Government Analysis Function.

A suite of PySpark, Pandas, and general pipeline utils for ONS projects.

A Python package for text-classification tasks associated with the production of official statistics, utilising semantic search and Retrieval Augmented Generation

ONS Map templates

Data repository for SDGs

The official repository of all publication ready ONS charts.

Beeswarm plot templates

Reusable chart templates for Svelte projects.

A Python package for creating accessible charts following Government Analysis Function guidance

A tool for exploring neighbourhood level Census data tables on a map.

Django Wagtail CMS for managing and publishing content for the Office for National Statistics (ONS)

This repository contains CROW, the Clerical Resolution Online Widget, an open-source project designed to help data linkers with their clerical matching needs!

ONS Dissemination development team documentation.

Repository to store code for outputs that are produced from the Subnational Statistics and Analysis division.

Python and PySpark implementation of Goldstein et al.'s Scalelink method of data linkage.

Starter page templates using ONS svelte-components.

AIMS 1 Scala API

An application for authoring eQ questionnaires.

For any specific queries or detailed feedback, please contact us at [email protected]

User interface for Respondent Account Services

This data set family supports the Explore Subnational Statistics initiative of the Office for National Statistics and provides low level geography data across many indicators for the United Kingdom's four nations. For any specific queries or detailed feedback, please contact us at [email protected]

This microservice manages surveys.

Legacy headless CMS used by the ONS website

This microservice manages collection exercises.

Address Index Matching Service Spark Job

ONS SDC Method for Cell Key Perturbation in R

A package for cleaning, validating, extracting and contextualising addresses with PySpark.

Resources and templates for documenting analytical workflows, assumptions and limitations.

The ONS respondent account service 'collection instrument' micro service.

Svelte components for ONS projects

Dataset API for customise my data

Secure Messaging Service for Respondent Account Services.

A template for scrolly projects with a .docx and ArchieML parser

A Python package containing a set of functions used to expedite and streamline the data linkage process.

A basic Python template to jump start a Python project

Automated Data Checker for Python

Digital publishing resumable file upload service that handles on the fly encryption and writing to S3. It updates images through the CMS

Go module implementing a kafka client wrapper, abstracting messages to go channels

ONS Address Index Matching Service containerised deployment for external users

A Flask application, designed to gather and report information on technologies used within ONS.

RAP minimum standard guidance for the Office for National Statistics

An internally developed Digital Landscape platform that presents the organisation’s repositories, projects, technology stack and GitHub Copilot usage.

Controller for handling datasets on the ONS website

Routes website requests to the appropriate apps

A helpful command line tool to make Digital Publishing developers' lives better

Search API on to the ONSWebsite Search engine

Scripts for generating common geography files.

An API for validating survey schemas

This microservice manages cases.

Common code for SDC services that use JWE

Scripts for translating eq-survey-runner surveys

Template that can be used when creating a new prototype-kit project.

A collection of logging libraries to implement standardised logging

An AWS Lambda Function to collect historic GitHub Copilot usage metrics from the GitHub API.

Python script to collect data on repositories within the organisation to be used on the Digital Landscape.

An API for the Tech Audit Tool, designed to gather and report information on technologies used within ONS.

Fork of WorldPop's repo for Malawi census support

Get data from the landregistry sparql endpoint and process it

A dashboard that displays useful information from multiple GCP projects.

A dashboard to monitor SDC survey response rates

An HTTP service for the controlling of search query

A collection of api clients written in go and used for the ONS website

A project to assist in composing multiple DP services

The Survey Assist API - Used to access backend data processing services such as classification

This microservice manages the survey sample.

R package for ONS ggplot2 themes and scales.

Github actions to format CPI items data

Test components in isolation using Godog / Gherkin

User documentation for the ONS Statistical Methods Library

A web page for checking the accessibility of spreadsheets

Templates for tables created by the data visualisation team, including static, interactive, and chables.

Repository for work-in-progress and infrequently used chart templates.

Common resources for ONS charts

Docker containers for Concourse CI

SIC Classification Utilities

SOC Classification Utilities

A web application that provides insights into GitHub organisations to make managing them easier.

Posts Slack alerts from status feeds.

An API used to manage the authorisation of users accessing data publishing services.

A benchmarking tool for EQ Survey Runner

A service for managing the retrieval of files from public and private s3 buckets

API for managing access control permissions for Digital Publishing API resources

A repository for the code supporting the paper "Practical Decompositions of Mean-Spliced on Published Indices"

Event driven nomad deployer service.

Schema definitions for the ONS system ecosystem

A prototyping kit that uses the ONS Design System

Library of classification functionality associated with UK SOC (Standard Occupational Classification)

Python client for interacting with our Blaise REST API.

React component implementations of the ONS Design System

An ONS branded theme for MkDocs based on Material for MkDocs

A tool to automatically populate the GDD CPD log from Cambridge Sparks OTJ log.

Static File Publisher

This is a Lambda function for the weekly retrieval of GitHub usernames and Organisation verified email addresses.

A Python package used to interact with the GitHub RESTful API

RAP-RPG

Reusable workflows for the Data Science Campus GitHub organisation.

A workflow to automate module updates in terraform projects

Terraform module for Google Cloud Privileged Access Manager (PAM) CI entitlements. Provisions a JIT elevation entitlement and the required PAM service agent binding for a target service account, with configurable approvers and grant duration.

Bootstrap module to support the use of terraform template

Python SDK for GitHub REST and GraphQL APIs, built for government organisations. Provides tools for org administration, repository governance, issue operations, and workflow automation.

Minimal repository template

ONS Data Scientist Learning Pathway

Data processing for the ONS Visual Journalism catalogue of releases.

An open-source orchestration toolkit for building and running reproducible Python data pipelines, developed by the Technology, Design and Innovation team at the Office for National Statistics.

Results processing methods for ONS business surveys

The dis-bundle-api is a backend service for managing and publishing datasets and content as bundles, similar to Florence’s collections.

React implementation of the ONS Design System (work in progress).

Configuration for cloud foundry UAA

Frontend service for deploying questionnaire packages to Blaise

Performance Test Scripts

Website to showcase the work of ONS

A registry of questionnaire schemas for eq-questionnaire-runner.

The new version of the Secure Messaging Service for Respondent Account Services.

Exploring agent based modelling in Python and packages that facilitate it.

Template for robo articles

A live editor for RosaeNLG PUG templates.

A Svelte Kit template for creating embedded semi-automated journalism content to sit within an iframe.

Architecture artifacts around the EMM metadata model, updated regularly

Repo for the eQ team to create prototypes using the ONS Design System

UI for the TO to launch into CATI questionnaire cases.

This project identifies outliers in small-area Gross Disposable Household Income (GDHI) data at the Lower Layer Super Output Area (LSOA) level and applies adjustments to ensure the statistics are reliable and suitable for publication.

Generates the static HTML for https://developer.ons.gov.uk/

Mock EQ for rasrm dev testing

Smoke tests for the dataset catalogue

This repository is for the Python driven Management UI application for the CIR Converter service

An API for handling file meta data

react component ui for table builder

Dashboard for managing Blaise users.

Enables greater flexibility in creating journeys through the website

User Interface that compliments survey-assist-themes

PoC using the themeFinder package from i.Ai to analyse responses

Repository holding tools and code snippets relating to all units on the Analysis for Action platform

Library of classification functionality associated with UK SIC (Standard Industrial Classification)

Renders a table in multiple formats, given data in json format

Copy of Census 2021 UI Prototype inside ONS Digital Org

A repository for the code supporting the paper "Are Multilateral Methods the Holy Grail of Price Measurement? A Critical Examination of Their Promises and Practical Performance in Dynamic Datasets", covering the perturbation test used in the paper.

PII validation library for Survey Assist

Utilities used as part of Survey Assist API or UI

Welcome to the Dissemination Engineering Standards repository

User interface for accessing Survey Assist backend

Tracks CMD dataset imports, recording the completion of the import steps to the Dataset API and then updating the import API job to state to completed.

Dynamic population modelling using Template Model Builder

A Python utility used to archive old, unused GitHub repositories from an organisation.

Analysis for the coding in analysis and research survey.

WIP for a python package to wrap up Govcookiecutter Templates into Zip files to make it easier for updating version in locked down environments

A minimalist demonstration repo to facilitate trying out the ClassifAI package via myBinder

Python pipeline classifying free-text survey responses into ISCO/ISIC schemes

A sandbox repo for the Fintrans project

Take screenshots of all ONS visualisations

The Pandemic Preparedness Toolkit (PPT) is a five-year project (2023 – 2028), funded by Wellcome, which aims to co-create a sustainable, online Toolkit that will build capacity for infectious disease surveillance in National Statistical Offices (NSOs).

Definitions and specifications for the 2031 Census EQ interfaces

Census 2031 FWMT: census31-fwmt-tm-mock (seeded from 2021, FMT-4)

Census 2031 FWMT: census31-fwmt-gateway-version-tracker (seeded from 2021, FMT-4)

Census 2031 FWMT: census31-fwmt-csv-service (seeded from 2021, FMT-4)

Housing affordability explorer

A Go API for management of redirects.

A poc for search as you type input using Survey Assist and the ONS Design System

Get mortgage interest rates from the Bank of England api

Census 2031 FWMT: census31-fwmt-fulfillment-event-service (seeded from 2021, FMT-4)

A Go API for data migration.

Action to scrape CPI weights for use in the ONS inflation calculator

Data Admin UI

Terraform infrastructure to generate load test against eq-survey-runner

Census 2031 FWMT: census31-fwmt-acceptance-tests (seeded from 2021, FMT-4)

Census 2031 FWMT: census31-fwmt-job-service (seeded from 2021, FMT-4)

Search Test bed

Census 2031 FWMT: census31-fwmt-outcome-service (seeded from 2021, FMT-4)

Release Calendar frontend controller

Controller for handling homepage on ONS website

To allow user feedback on the ONS website

Supplementary Data Service

An API for validating survey schemas

Svelte code for RTI weekly dashboard

RM Census acceptance tests

A rendering library for Dissemination frontend go microservices. dis-design-system-go contains template, localisations, model structs, css and javascript that are core to all dissemination frontend services.

An message adapter service for the Census RM services to communicate with the FWMTG

Shared/common classes which are used to validate sample data for Survey Collection & Ingestion

A registry of census questionnaire schemas for census31-eq-questionnaire-runner.

A benchmarking tool for Census EQ Questionnaire Runner

A generic vector store and api used by survey assist. This can deploy either an industry, occupation or search as you type vector store

A scheduler application for automating the purging of cache for paths to be published.

A FastAPI service for generating and exporting, ONS Design System chart images

Publish file metadata to Sub/Sub topic when uploaded to bucket.

Proof-of-concept MCP server allowing LLM chatbots to access the Explore Local Statistics API

This project is Collection Instrument Registry. It will manage the storage and versioning of Electronic Questionnaires used but the EQ services.

sds-common is a Python library for common functionality used to interact with SDS

A new version of `eq-publisher` for translating questionnaires built in Author into Runner V3 format

Legacy part of the ONS website

Cloud Functions to support CMA

Generate accessible ODS spreadsheets with in a web browser or using Node.js

Shared classification library, LLM interface and LLM prompts for Survey Assist

Search Reindexing Batch Job

Service to store searchable content into elasticsearch

Service to retrieve data to update search index

Small scale survey generation for wireframes, survey design or proof of concepts. Not for delivering surveys at scale for publication.

Frontend svelte app for ONS visual journalism's searchable release catalogue

An interactive tool to allow users to adjust and explore population projections for the UK.

Proof of concept for using svelteplot library to replace svelte-charts for advanced data visualisation products

Prototype for building census profiles for sub-population groups. Based on synthetic data.

A Svelte Kit scrollytelling template designed to run from RosaeNLG PUG templates, as generated using [this editor](https://github.com/ONSvisual/robo-editor)).

Scrolly project on the intersectionality of disability, unemployment and ethnicity.

Utilities for robo-journalism.

Generate accessible XLSX spreadsheets in JS

A questionnaire launcher for eq-survey-runner

Scripts for translating eq-questionnaire-runner surveys

A service for loading data into the Supplementary Data Service (SDS).

This repository contains the locust performance testing components for the SDS application.

Automated quality assurance methods for demographic outputs

A boilerplate for creating fastapi services

Census 2031 FWMT: census31-fwmt-common (seeded from 2021, FMT-4)

JS library for client side token renewal

A questionnaire launcher for census31-eq-survey-runner

Node.js client for our Blaise UAC Service

HTML dashboard produced using Quarto displaying data from the Office for National Statistics’ Health Insight Survey, commissioned by NHS England.

Provides a GET endpoint to generate and download files

Fork of davidcarboni/cryptolite

Reusable Python logging library implementing the ONS Dissemination JSON log standard for Lambda functions

API for managing the release calendar

Using the table templates to prototype the ELS Area Comparison Page

Microservice to automatically process sample jobs

A simple FastAPI to mock the SDS service required by eq-runner.

A wrapper for govulncheck to allow exclusions

eQ Lookup Suggestions Data

Network utilities library

Common embedding functionality used by Survey Assist

A Go Service to redirect legacy URLs requested by users

Python Jupyter Notebook for GCP log investigation

Cloud Function to perform questionnaire data point-in-time restore

Census 2031 FWMT: census31-fwmt-canonical (seeded from 2021, FMT-4)

Census 2031 FWMT: census31-fwmt-performance-tests (seeded from 2021, FMT-4)

Census 2031 FWMT: census31-fwmt-perf-msg-builder (seeded from 2021, FMT-4)

Census 2031 FWMT: shared Maven parent BOM

Census 2031 FWMT: census31-fwmt-events (seeded from 2021, FMT-4)

Census 2031 FWMT: census31-fwmt-storage-utils (seeded from 2021, FMT-4)

An experimental linter for use when working with Go templates that construct JSON snippets.

A Golang microservice that expands industry codes, output area codes, identifies survey codes and years in the search query and returns them separately

Fake upstream search stub to mimic upstream services using a generic search contract to integrate with our search stack

User Interface for respondents to access ONS Survey Data Collection questionnaires and services

A vault library for Go.

Check desired apps are running on both web_mount boxes and report any issues to Slack

A simple FastAPI to mock the CIR service required by eq-runner

This project is demo of design system implemenented in Python and Flask.

Repository for Google Cloud Functions used with SDS

A general purpose static website generator that can render static content and optionally content pulled from a Craft CMS instance.

A generic interface to a vector store used for SOC classification

This contains the code for the dp feedback api

Data pipeline for ONS Beta website

Functions to copy questionnaire data from NISRA SFTP to GCP storage bucket.

REST API for managing cache control information for pages within the legacy CMS

A Java client for the dp-dataset-api service

A Java client library for the dp-image-api

A generic interface to a vector store used for SIC classification

Spring Boot Microservice for AIMS bulk match solution

A registry for supplementary dataset schemas for the SDS service.

Azure Pipeline to backup Blaise users from local SQLite DB

A python client for the blaise-data-delivery-status API

Cloud functions for the Blaise editing service

Send logging messages from GCP to Slack

Services for interacting with Totalmobile's APIs.

Lightweight Node.js provider for Google IAP ID tokens

This repository contains functionality for checking questionnaire survey days and automating daybatch creation.

Node.js client for our Blaise REST API

Slack alerts for data delivery issues

Cloud Function for processing receipts from NiFi

Image importer

Periodic nomad job for checking integrity of zebedee workspace

Proxy for handling the cache for pages within the legacy CMS

Digital Publishing image management API

This is a demo R pipeline for to use the `crvs.pkg` functionality to process and analyse data into publication ready tables.

R package for the production of vital statistics from civil registration data. The package handles data validation, processing/derivation, aggregation and analysis.

Public repository for the ESPRESSO Python pipeline

A benchmarking tool for RASRM

A java client library for sending slack messages

A Java JSON Web Token (JWT) verification library

Encrypted HTTP Multipart Upload

Access Java resources as .properties, File, XML or String

Fork of davidcarboni/httpino

Brutally opinionated REST micro-framework.

Address Index Acceptance tests

tidysheet takes messy Excel data and converts it to tidy data with a single value per row.

This repository holds the code for the data-api bundle scheduler, which is used to automate the publication of datasets included within a bundle.

A working-level exercise in data linkage, demonstrating how to clean and link data in Python

An interactive linkage notebook. This goes through an entire linkage pipeline, encouraging participants to alter match thresholds and variable weighting to maximise linkage quality.

This project is demo of design system implemenented in Python and Flask.

Consumes error messages from the CMD import process and report them to the Dataset API

Routes API requests to set services, with additional header tags

GitHub Desktop tutorial repository

CPI items interactive

A tool used to audit GitHub Organisations for compliance with ONS' GitHub Usage Policy.

A series of functions used to check adherence to the GitHub Usage Policy.

A template repository for all new KEH repositories.

Flask survey from config PoC

ETL template that deploys results to MkDocs GitHub pages.

A basic Python template to jump start a Python project

A basic Python template to jump start a Python project

A basic ETL pipeline Python template to jump start a Python project

A basic Python template to jump start a Python project