UK Public Sector Code by Organisation

A cookiecutter template for data science projects within His Majesty's Government and wider public sector.

Guidance for quality assurance of code for civil service researchers and analysts.

R package: generate best-practice stats spreadsheets for publication

Good Practice Tables - an XlsxWriter wrapper to write consistently formatted statistical tables to Excel.

ONS Design System

Development website for collecting and disseminating UK data for the Sustainable Development Goal global indicators.

Reusable map components for Svelte projects.

afcharts is an R package for creating accessible plots by the Government Analysis Function.

A suite of PySpark, Pandas, and general pipeline utils for ONS projects.

A Python package for text-classification tasks associated with the production of official statistics, utilising semantic search and Retrieval Augmented Generation

Data repository for SDGs

The official repository of all publication ready ONS charts.

A tool for exploring neighbourhood level Census data tables on a map.

Statistical Methods Library

A Python package for creating accessible charts following Government Analysis Function guidance

Django Wagtail CMS for managing and publishing content for the Office for National Statistics (ONS)

This repository contains CROW, the Clerical Resolution Online Widget, an open-source project designed to help data linkers with their clerical matching needs!

Statistical Methods Library for Python Pandas methods used in SPP.

ONS Dissemination development team documentation.

Repository to store code for outputs that are produced from the Subnational Statistics and Analysis division.

Python and PySpark implementation of Goldstein et al.'s Scalelink method of data linkage.

AIMS 1 Scala API

An application for authoring eQ questionnaires.

Starter page templates using ONS svelte-components.

User interface for Respondent Account Services

For any specific queries or detailed feedback, please contact us at [email protected]

Creates a frequency table and performs cell-key perturbation

Website cataloguing Statistical Methods Library

This data set family supports the Explore Subnational Statistics initiative of the Office for National Statistics and provides low level geography data across many indicators for the United Kingdom's four nations. For any specific queries or detailed feedback, please contact us at [email protected]

ONS SDC Method for Cell Key Perturbation in R

This microservice manages surveys.

This microservice manages collection exercises.

A package for cleaning, validating, extracting and contextualising addresses with PySpark.

Legacy headless CMS used by the ONS website

Address Index Matching Service Spark Job

Resources and templates for documenting analytical workflows, assumptions and limitations.

Dataset API for customise my data

A basic Python template to jump start a Python project

Svelte components for ONS projects

The ONS respondent account service 'collection instrument' micro service.

Secure Messaging Service for Respondent Account Services.

Automated Data Checker for Python

Digital publishing resumable file upload service that handles on the fly encryption and writing to S3. It updates images through the CMS

Go module implementing a kafka client wrapper, abstracting messages to go channels

ONS Address Index Matching Service containerised deployment for external users

Calculating statistics for national and regional research and development expenditure

A Flask application, designed to gather and report information on technologies used within ONS.

RAP minimum standard guidance for the Office for National Statistics

An internally developed Digital Landscape platform that presents the organisation’s repositories, projects, technology stack and GitHub Copilot usage.

An API for validating survey schemas

Controller for handling datasets on the ONS website

A helpful command line tool to make Digital Publishing developers' lives better

Search API on to the ONSWebsite Search engine

Routes website requests to the appropriate apps

This microservice manages cases.

Common code for SDC services that use JWE

Scripts for translating eq-survey-runner surveys

Scripts for generating common geography files.

Template that can be used when creating a new prototype-kit project.

A collection of logging libraries to implement standardised logging

A Python package containing a set of functions used to expedite and streamline the data linkage process.

A mongo db library for Go.

An API for the Tech Audit Tool, designed to gather and report information on technologies used within ONS.

An AWS Lambda Function to collect historic GitHub Copilot usage metrics from the GitHub API.

Python script to collect data on repositories within the organisation to be used on the Digital Landscape.

Fork of WorldPop's repo for Malawi census support

Github actions to format CPI items data

A dashboard that displays useful information from multiple GCP projects.

Get data from the landregistry sparql endpoint and process it

Templates for tables created by the data visualisation team, including static, interactive, and chables.

A service for managing the retrieval of files from public and private s3 buckets

API for managing access control permissions for Digital Publishing API resources

An HTTP service for the controlling of search query

SIC Classification Utilities

A project to assist in composing multiple DP services

This microservice manages the survey sample.

A repository for the code supporting the paper "Practical Decompositions of Mean-Spliced on Published Indices"

Event driven nomad deployer service.

Docker containers for Concourse CI

An API used to manage the authorisation of users accessing data publishing services.

A web application that provides insights into GitHub organisations to make managing them easier.

SOC Classification Utilities

Schema definitions for the ONS system ecosystem

The Survey Assist API - Used to access backend data processing services such as classification

A prototyping kit that uses the ONS Design System

Test components in isolation using Godog / Gherkin

Posts Slack alerts from status feeds.

Library of classification functionality associated with UK SOC (Standard Occupational Classification)

A benchmarking tool for EQ Survey Runner

A dashboard to monitor SDC survey response rates

Python client for interacting with our Blaise REST API.

React component implementations of the ONS Design System

User documentation for the ONS Statistical Methods Library

An ONS branded theme for MkDocs based on Material for MkDocs

Repository for work-in-progress and infrequently used chart templates.

Common resources for ONS charts

A collection of api clients written in go and used for the ONS website

A tool to automatically populate the GDD CPD log from Cambridge Sparks OTJ log.

Static File Publisher

This is a Lambda function for the weekly retrieval of GitHub usernames and Organisation verified email addresses.

A Python package used to interact with the GitHub RESTful API

An application to track an organisation's compliance with ONS Usage Policy

AI Assist PoC (initially for TLFS)

Response rate dashboard for the Coding in Analysis and Research Survey (CARS) wave 3.

Bootstrap module to support the use of terraform template

ONS Data Scientist Learning Pathway

Data processing for the ONS Visual Journalism catalogue of releases.

This repository is for the Python driven Management UI application for the CIR Converter service

UI for the TO to launch into CATI questionnaire cases.

Exploring agent based modelling in Python and packages that facilitate it.

react component ui for table builder

Dashboard for managing Blaise users.

A registry of questionnaire schemas for eq-questionnaire-runner.

The new version of the Secure Messaging Service for Respondent Account Services.

A live editor for RosaeNLG PUG templates.

Configuration for cloud foundry UAA

Smoke tests for the dataset catalogue

The dis-bundle-api is a backend service for managing and publishing datasets and content as bundles, similar to Florence’s collections.

An API for handling file meta data

Enables greater flexibility in creating journeys through the website

Performance Test Scripts

User Interface that compliments survey-assist-themes

PoC using the themeFinder package from i.Ai to analyse responses

Mock EQ for rasrm dev testing

Website to showcase the work of ONS

Repository holding tools and code snippets relating to all units on the Analysis for Action platform

Frontend service for deploying questionnaire packages to Blaise

Library of classification functionality associated with UK SIC (Standard Industrial Classification)

Repo for the eQ team to create prototypes using the ONS Design System

Renders a table in multiple formats, given data in json format

Copy of Census 2021 UI Prototype inside ONS Digital Org

A repository for the code supporting the paper "Are Multilateral Methods the Holy Grail of Price Measurement? A Critical Examination of Their Promises and Practical Performance in Dynamic Datasets", covering the perturbation test used in the paper.

PII validation library for Survey Assist

Utilities used as part of Survey Assist API or UI

This project identifies outliers in small-area Gross Disposable Household Income (GDHI) data at the Lower Layer Super Output Area (LSOA) level and applies adjustments to ensure the statistics are reliable and suitable for publication.

Welcome to the Dissemination Engineering Standards repository

Logging library for Go

User interface for accessing Survey Assist backend

Tracks CMD dataset imports, recording the completion of the import steps to the Dataset API and then updating the import API job to state to completed.

Dynamic population modelling using Template Model Builder

Generates the static HTML for https://developer.ons.gov.uk/

A Python utility used to archive old, unused GitHub repositories from an organisation.

Analysis for the coding in analysis and research survey.

WIP for a python package to wrap up Govcookiecutter Templates into Zip files to make it easier for updating version in locked down environments

RAP-RPG

A workflow to automate module updates in terraform projects

Python pipeline classifying free-text survey responses into ISCO/ISIC schemes

Proof of concept for auto-documentation of code using LLMs.

A sandbox repo for the Fintrans project

Take screenshots of all ONS visualisations

Census 2031 FWMT: census31-fwmt-csv-service (seeded from 2021, FMT-4)

Get mortgage interest rates from the Bank of England api

A benchmarking tool for Census EQ Questionnaire Runner

RM Census acceptance tests

Data Admin UI

Action to scrape CPI weights for use in the ONS inflation calculator

Search Test bed

Census 2031 FWMT: census31-fwmt-fulfillment-event-service (seeded from 2021, FMT-4)

A registry of census questionnaire schemas for census31-eq-questionnaire-runner.

Terraform infrastructure to generate load test against eq-survey-runner

Scripts for translating eq-questionnaire-runner surveys

A rendering library for Dissemination frontend go microservices. dis-design-system-go contains template, localisations, model structs, css and javascript that are core to all dissemination frontend services.

An API for validating survey schemas

A Go API for management of redirects.

Census 2031 FWMT: census31-fwmt-job-service (seeded from 2021, FMT-4)

A Go API for data migration.

To allow user feedback on the ONS website

Controller for handling homepage on ONS website

Release Calendar frontend controller

A wrapper for govulncheck to allow exclusions

Census 2031 FWMT: census31-fwmt-outcome-service (seeded from 2021, FMT-4)

Legacy part of the ONS website

Automated quality assurance methods for demographic outputs

eQ Lookup Suggestions Data

A questionnaire launcher for census31-eq-survey-runner

Network utilities library

This repository contains the locust performance testing components for the SDS application.

Supplementary Data Service

Common embedding functionality used by Survey Assist

A Go Service to redirect legacy URLs requested by users

Python Jupyter Notebook for GCP log investigation

Cloud Function to perform questionnaire data point-in-time restore

A FastAPI service for generating and exporting, ONS Design System chart images

Census 2031 FWMT: census31-fwmt-acceptance-tests (seeded from 2021, FMT-4)

A scheduler application for automating the purging of cache for paths to be published.

HTML dashboard produced using Quarto displaying data from the Office for National Statistics’ Health Insight Survey, commissioned by NHS England.

Census 2031 FWMT: census31-fwmt-canonical (seeded from 2021, FMT-4)

Census 2031 FWMT: census31-fwmt-performance-tests (seeded from 2021, FMT-4)

Census 2031 FWMT: census31-fwmt-perf-msg-builder (seeded from 2021, FMT-4)

Census 2031 FWMT: shared Maven parent BOM

Census 2031 FWMT: census31-fwmt-events (seeded from 2021, FMT-4)

Census 2031 FWMT: census31-fwmt-common (seeded from 2021, FMT-4)

Census 2031 FWMT: census31-fwmt-storage-utils (seeded from 2021, FMT-4)

Census 2031 FWMT: census31-fwmt-tm-mock (seeded from 2021, FMT-4)

This project is Collection Instrument Registry. It will manage the storage and versioning of Electronic Questionnaires used but the EQ services.

Utilities for robo-journalism.

A generic vector store and api used by survey assist. This can deploy either an industry, occupation or search as you type vector store

sds-common is a Python library for common functionality used to interact with SDS

Generate accessible XLSX spreadsheets in JS

A service for loading data into the Supplementary Data Service (SDS).

A poc for search as you type input using Survey Assist and the ONS Design System

An experimental linter for use when working with Go templates that construct JSON snippets.

Service to store searchable content into elasticsearch

Service to retrieve data to update search index

A Golang microservice that expands industry codes, output area codes, identifies survey codes and years in the search query and returns them separately

Fake upstream search stub to mimic upstream services using a generic search contract to integrate with our search stack

User Interface for respondents to access ONS Survey Data Collection questionnaires and services

Census 2031 FWMT: census31-fwmt-gateway-version-tracker (seeded from 2021, FMT-4)

Svelte code for RTI weekly dashboard

JS library for client side token renewal

A vault library for Go.

Check desired apps are running on both web_mount boxes and report any issues to Slack

Definitions and specifications for the 2031 Census EQ interfaces

A questionnaire launcher for eq-survey-runner

A simple FastAPI to mock the CIR service required by eq-runner

A simple FastAPI to mock the SDS service required by eq-runner.

This project is demo of design system implemenented in Python and Flask.

Repository for Google Cloud Functions used with SDS

A general purpose static website generator that can render static content and optionally content pulled from a Craft CMS instance.

A generic interface to a vector store used for SOC classification

This contains the code for the dp feedback api

Data pipeline for ONS Beta website

Functions to copy questionnaire data from NISRA SFTP to GCP storage bucket.

REST API for managing cache control information for pages within the legacy CMS

A Java client for the dp-dataset-api service

A Java client library for the dp-image-api

A generic interface to a vector store used for SIC classification

Shared classification library, LLM interface and LLM prompts for Survey Assist

Spring Boot Microservice for AIMS bulk match solution

Microservice to automatically process sample jobs

Shared/common classes which are used to validate sample data for Survey Collection & Ingestion

A registry for supplementary dataset schemas for the SDS service.

Azure Pipeline to backup Blaise users from local SQLite DB

A python client for the blaise-data-delivery-status API

Cloud functions for the Blaise editing service

Cloud Functions to support CMA

Send logging messages from GCP to Slack

Services for interacting with Totalmobile's APIs.

Lightweight Node.js provider for Google IAP ID tokens

This repository contains functionality for checking questionnaire survey days and automating daybatch creation.

Node.js client for our Blaise UAC Service

Node.js client for our Blaise REST API

Slack alerts for data delivery issues

Cloud Function for processing receipts from NiFi

Publish file metadata to Sub/Sub topic when uploaded to bucket.

Image importer

Periodic nomad job for checking integrity of zebedee workspace

Proxy for handling the cache for pages within the legacy CMS

Digital Publishing image management API

Provides a GET endpoint to generate and download files

API for managing the release calendar

Search Reindexing Batch Job

This is a demo R pipeline for to use the `crvs.pkg` functionality to process and analyse data into publication ready tables.

R package for the production of vital statistics from civil registration data. The package handles data validation, processing/derivation, aggregation and analysis.

Public repository for the ESPRESSO Python pipeline

A benchmarking tool for RASRM

A java client library for sending slack messages

A Java JSON Web Token (JWT) verification library

Encrypted HTTP Multipart Upload

Access Java resources as .properties, File, XML or String

Fork of davidcarboni/httpino

Brutally opinionated REST micro-framework.

Address Index Acceptance tests

A boilerplate for creating fastapi services

tidysheet takes messy Excel data and converts it to tidy data with a single value per row.

Generate accessible ODS spreadsheets with in a web browser or using Node.js

Proof of concept for using svelteplot library to replace svelte-charts for advanced data visualisation products

Fork of davidcarboni/cryptolite

This repository holds the code for the data-api bundle scheduler, which is used to automate the publication of datasets included within a bundle.

A working-level exercise in data linkage, demonstrating how to clean and link data in Python

An interactive linkage notebook. This goes through an entire linkage pipeline, encouraging participants to alter match thresholds and variable weighting to maximise linkage quality.

Frontend svelte app for ONS visual journalism's searchable release catalogue

This project is demo of design system implemenented in Python and Flask.

The repo is the terraform to setup author in GCP.

A new auth service stub to ensure we can test all actions associated with login/authentication

Dissemination client library for Redis or similar protocols.

A Java Client Library for the dp-files-api

Consumes error messages from the CMD import process and report them to the Dataset API

The Pandemic Preparedness Toolkit (PPT) is a five-year project (2023 – 2028), funded by Wellcome, which aims to co-create a sustainable, online Toolkit that will build capacity for infectious disease surveillance in National Statistical Offices (NSOs).

GitHub Desktop tutorial repository

CPI items interactive

A tool used to audit GitHub Organisations for compliance with ONS' GitHub Usage Policy.

A series of functions used to check adherence to the GitHub Usage Policy.

A template repository for all new KEH repositories.

Flask survey from config PoC

test