Description#
We propose mechanisms for wholesale reimplementations of library functions.
This would allow groups outside of, say, networkx to (a) provide new
functions to replace parts of networkx, or (b) provide data structures
that can pass through NetworkX’s existing computational pipelines.
This SPEC focuses on the rationale for these mechanisms, and provides links to implementations related technical discussions.
Core Project Endorsement#
Ecosystem Adoption#
Implementation#
A successful prototype implementation of this SPEC (at least in spirit) is already available inside NetworkX.
NetworkX has developed a dispatching layer which can plug into multiple backends. Currently we have backends which can dispatch to CuGraph, GraphBLAS and a joblib bpacked parallel implementation of algorithms in NetworkX.
One of the goals here is to provide drop in replacement for old code written with NetworkX. Users should be able to set a config option/env variable and dispatch their code to different backends. These backends could be hardware specific ones, reimplementations in other languages (hello rust!) or using a totally new data structure (GraphBLAS in the case of NetworkX).
On the NetworkX side we are still ironing out more details, and we are still missing out on a detailed spec (or SPEC). I think other projects in the scientific python ecosystem could also benefit from some kind of dispatching along these lines. There is some progress towards creating an engine API in scikit-learn which follows similar rational.