Zcash Observatory

Motivation

As Zcash adoption expands and the network scales, consumers, researchers, and businesses will all require high-quality data quantifying network health and security. Users need data and visibility related to both performance and safety, so observatory nodes will detect and alert for a variety of miner & network phenomena. Additionally, they unlock the ability to study the peer-to-peer network's topology and latency characteristics, which will inform protocol design decisions related to Zcash scaling and privacy. Implementing observatory NetSec tools will increase Zcash’s attractiveness for commercial adoption and infrastructure integration.

Key deliverables & features:

We will modify the existing Zcash codebase (https://github.com/zcash/zcash) to add “observatory” features, which collect and archive key ephemeral off-chain data that are currently not retained. A single observatory node can detect and alert on several network/security phenomena including: deep reorganizations, double spends, and selfish mining.

The observatory logging and alerting features will be contributed into the main codebase, so that any Zcash node operator can opt-in to keeping & contributing logs. Archival features can be implemented through a mix of AMQP, wrappers, and custom modifications.

Observatory data features:

  • Multiple precise timestamp types for transactions and blocks
    • Block/transaction receipt at peer-to-peer layer
    • Block/transaction verification time and timestamp
    • Block/transaction notification from each peer
  • Alternative/orphaned blocks and transactions
    • Cross-compare difficulty and contents of the main blocks and alternative blocks
    • For transparent transactions, can detect probable double spends
    • For shielded transactions, can compare nullifiers for possible double spends
    • With advanced timestamps, enables detection of stubborn & selfish mining
  • Garbage collection - retain invalid data and everything thrown at the node

Implementing local observatory capabilities is a first step that unlocks global Zcash NetSec monitoring. We plan to expand upon this project by integrating the Zebra network stack and building public utilities including: a data aggregation pipeline enabling global analyses (e.g. network topology, latency, homogeneity, etc), a queryable database for researchers, and a dashboard designed for both technical and semi-technical users. You can view the long-term roadmap here: fellows.link/ObservatoryRoadmap

Execution risks

Observatory back-end and database development is relatively straightforward, and observatory features have been previously executed by Mitchell for other decentralized systems (Loki & Monero). While there are still some design decisions around how data will be piped, no intractable obstacles are expected. Variability mainly enters when we begin iterative testing with various stakeholders, including: engineers building real-time monitoring systems, researchers informing decisions with accessible high-quality data, and less-technical users who may wish to explore historical and/or live data.

Applicant background

Principal Investigator: Mitchell Krawiec-Thayer, Ph.D.

Lead Engineer: Pranav Thirunavukkarasu

  • Blockchain Engineer, Full-Stack Developer at Insight
  • Built a DevOps tool to spin up various local Lightning Network clusters for development and network testing purposes: fellows.link/pranav_video
  • Developed a Full-Stack Ethereum mobile wallet focused on user experience, wallet security, and network reliability
  • Full-stack engineering experience at Hewlett Packard Enterprise, focused on front-end user interfaces and low-level networking

Budget and justification

Item Cost Note
Full-time SWE Developer-In-Residence $25k Pranav
Part-time mentoring from archival expert $5k Mitchell
Insight overhead $7.5k AWS, code audits, office, etc

Tipping

Tips Received
???  
ZEC

Campaign

Started
a month ago
Funding
$37,500 
Funded through  
1
Fork detection and detailed timestamps
2
Full observatory node & documentation

Fork detection and detailed timestamps

Estimate: April 2020
Reward: $18,750
Observatory alerts users of forks it detects as well as logging detailed timestamps from blocks (block seen time, block verified time, peer notified time, etc.)

Payment Request

The team may request a payout for this milestone at any time.