Using Shadow does require some Solidity knowledge, but you don't need to be an expert.

At a high-level, you need to be able to:

• Decide what you want your end shadow event data table to look like

• Read smart contracts well enough to understand where to insert shadow events

• Write short snippets of Solidity code to declare and emit shadow events

• Use ChatGPT tools to help you out when you get stuck (see recommendations below)

How long will it take me to learn?

If you're a non-Solidity engineer, it'll be easy. You'll be able to use Shadow within 1-2 hrs.

If you're a data analyst that doesn't know any Solidity at all, you'll be able to use Shadow effectively within a few days. Some shadow events are easier to write, so you may even be able to get things working within a few hours. Many people who primarily have a SQL skillset have learned enough to use Shadow effectively, so don't be deterred!

High-level steps

We'll walk you through the steps to get started with detailed instructions and screenshots, and also provide tips on common issues that may arise as you create your own shadow events.

(e.g. recommended tools and how to get contract code)

I. Get things set up

1. Set up recommended tools

We highly recommend Solidity beginners to use these tools in order to use Shadow effectively, and have written detailed instructions with screenshots on how to get them set up.

2. Open the contract in Cursor

See "How to use" for the .

• You can also download from Etherscan using Open In > Remix IDE from the Contract tab.

• Right click the folder with the contract address and select Download

• Open a new Cursor window > Open from folder> open the folder with the contract address

After this step, you should have a folder with the .sol file(s) from the contract imported into Cursor. Most contracts will have multiple folders, subfolders, and .sol files within each. This may look intimidating at first, but you'll usually only need to modify 1-2 files to add a shadow event.

II. Add your shadow event

1. Declare the shadow event within the contract

The first step of adding a shadow event is to declare it somewhere in the contract. This makes your contract aware of the event and its data parameters.

To keep things organized, you should try to declare your shadow event where the contract's other events are already declared. Search the codebase for "event" – it's usually pretty easy to find.

Declare your new shadow event where the contract's existing events are declared.

If you don't need to add a brand new event and instead just want to add data parameters to an existing event, you can do that too. If you're modifying an existing event, it's helpful for readability to rename it by pre-fixing "Shadow" to the name, e.g. ShadowMint

2. Identify the contract function to emit the shadow event in

The second step

3. Implement the shadow event emit within the contract function

4. Compile your shadow contract in Cursor using the Solidity extension

See "How to use" for the .

III. Deploy on Shadow and get data

1. Paste your shadow contract in the Shadow browser IDE and deploy

Go to the contract page on . The URL of the page should look something like this, with the contract address replaced and using your unique project_id at the end.

Click on the Open in editor button in the top right corner.

Copy paste your modified code into the right .sol files, and click Deploy. If you modified multiple .sol files to add your shadow event, be sure to replace the contents for each one.

2. Access your shadow event data

Once deployed, you can see your shadow events happening in realtime on the Event Feed tab.

You can also access your shadow event data via RPC, streams to your own database, or a Shadow hosted SQL API. Please talk to us if you're having trouble deciding or setting that up.

Get in touch

What is a shadow event?

A shadow event is an event log that only lives in a shadow contract, and not on the version of the contract that is deployed on a public chain. Shadow events are fully customizable, permissionless to write on any contract, don't cost any gas to emit, and can access any internal contract state.

Shadow events are a powerful way to access realtime data for analysis, indexing, and testing purposes, and can significantly reduce the number of data pipelines that you need to maintain.

You can access shadow events via standard methods (e.g. eth_getLogs) using a Shadow RPC, or you can have Shadow automatically decode and index the data for you to export. We currently offer hourly data exports in Parquet to GCP/S3, and are actively working on other export formats.

What is a shadow contract?

When you modify a contract in Shadow to include new shadow events, its bytecode is updated and replaced at the same contract address on your shadow fork. Shadow contracts have the same exact write (e.g. state-changing) functions as their counterparts deployed on the public chain, but can also include additional custom events and view functions that you add.

This means that as your shadow fork executes transactions that occur on the public chain, its state stays in sync, but it also has additional custom data accessible to you.

Unlike their publicly deployed counterparts, shadow contracts don't have any contract size limitations, can be easily updated at any time, and anyone can write their own version of one.

You can compile and deploy shadow contracts in one-click, and Shadow also handles factory created contracts (e.g. only need to deploy once) and proxy contracts for you.

What is a shadow fork?

Shadow forks are powerful private execution environments that can be spun up instantly in one click. They're designed to mirror the state of a public blockchain in realtime, but differ in a few important ways that improve data accessibility.

On public blockchains:

• Smart contracts are rigid by design and are hard to change once deployed which is great for security, but bad for data accessibility. This presents a lot of data and indexing problems downstream.

• Contracts have a 24,576 byte size limit, which often forces developers to choose between functionality and data accessibility.

• Generating semantic data (e.g. event logs) costs gas, which are paid by end users. These fees are several orders of magnitude greater than the cost of equivalent operations in a traditional software stack.

On shadow forks:

• Smart contracts are flexible and augmenting them is permissionless. Anyone can write a shadow implementation for any contract, at any time.

• People who need onchain data can get it precisely for their specific needs on-demand, and everyone else has nothing to worry about.

• Gas fees and contract size limits are removed. Projects can optimize their mainnet contracts and reduce gas fees for users by moving event logs and view functions to a shadow contract implementation, and make room for new functionality while significantly improving data accessibility.

What is a shadow node?

A shadow node is a client that makes data on a shadow fork accessible. You can make all of the standard RPC calls to shadow nodes in the same way you do normal nodes – all you need to do is swap out the RPC URL. Shadow hosts and manages shadow nodes for you.

We plan to open source shadow nodes in the future so anyone can run them locally, and have them work with node clients like Reth, Erigon, and Geth.

Other FAQs

Are you a beginner to Solidity?

Start with our step-by-step guide for detailed instructions with screenshots to get set up.

What chains does Shadow support?

Shadow is currently production ready for Ethereum Mainnet, as well as Arbitrum Mainnet with limited functionality. We plan to support Optimism and Base Mainnets by August 2024. If there are additional chains you need supported, please reach out to us at [email protected].

How is Shadow different than other forks? (e.g. Tenderly?)

Forking functionality offered by other developer tools, such as Tenderly or Anvil, are more useful for one-off simulations to test a sequence of transactions that you need to specify. Shadow is more useful for continuously simulating what is actually happening on mainnet, and generating additional data based on mainnet transactions. Tenderly forks do not continuously execute (i.e. “shadow”) mainnet transactions, and also does not support event backfills.

How far behind the chain tip are Shadow forks?

Your Shadow fork will fully execute and index shadow events a few seconds after a transaction is confirmed on mainnet (P95 <5s). This is fast enough to satisfy most realtime use cases. If you need even faster performance, please reach out to us so we can better understand your specific use case.

Can I arbitrarily set state or send a test transaction on Shadow forks?

No – the general idea behind Shadow is to really only "control" the contract and EVM read path by overriding contracts. For example, it's not currently designed to allow you to arbitrarily top up balances, or send one-off transactions. It's designed such that your Shadow fork can be an extension of mainnet, with essentially the same state, but emit a lot more data that you are able to fully define and can easily use.

Do I need to know Solidity in order to use Shadow?

Yes – you need to be at least somewhat comfortable with Solidity. Shadow is designed for Solidity developers, or analysts who are familiar enough with Solidity to read contract code and add event log code. You won’t need to write complete Solidity smart contracts in Shadow, but you need to be able to write snippets of Solidity code to add custom shadow events. We know that many onchain analysts are most familiar with SQL, and we’re investigating ways to make it easy for folks with very little Solidity experience to use Shadow effectively. See our guide for .

Could I put all of my contract’s events on Shadow instead of mainnet?

Yes – you can move all of your events off of mainnet and onto Shadow. Based on our research, this would result in users saving between ~4-12% in gas costs. Keep in mind, this is not strictly necessary in order to use Shadow – you can leave your mainnet events as-is and simply add new events on Shadow.

How does Shadow handle user data?

Shadow accesses and stores public blockchain data, or data that can be derived from public blockchain data. This includes: wallet addresses, wallet transactions, contract events, and contract storage slots. Otherwise, Shadow does not collect or store any personal identifiable information (PII) on your users. For example, we do not collect or store IP address, name, email address, locale, or any other sensitive user information.

Every shadow fork comes with its own Shadow RPC and a standard set of JSON-RPC endpoints. Shadow RPCs look and feel like a normal RPC, but contains additional shadow data that's custom to your shadow fork.

Each Shadow RPC is versioned, which allows you to safely deploy changes to your shadow fork and test them without disrupting previously running jobs.

Since your Shadow RPC behaves like a normal RPC, you can use it as a drop-in replacement in your own infrastructure (e.g. include shadow events in your custom indexer, or add shadow events to your self-hosted subgraph).

API keys

Shadow is a platform that reduces the engineering time and costs for advanced onchain data indexing and analytics. Shadow helps you get more data, save users gas, and ship products faster.

Our first product enables you to log anything you want on any smart contract, completely gaslessly.

We've built drop in production-grade infrastructure for shadow events that allows you to get custom onchain data with just a few lines of code.

What does Shadow do?

Shadow enables you to modify a deployed contract's source code to add gasless custom event logs and view functions on a that is instrumented to mirror mainnet state in realtime.

Shadow provides you the tools and infrastructure to:

• Spin up a new shadow fork in one click

• Augment any contract with custom event logs and view functions

• Instantly access your data via standard RPC methods

Shadow is a fully hosted platform. We manage lots of boring things for you, including:

• Nodes for both the shadow fork and public chain

• Keeping state in sync in realtime, including re-orgs

• Compiling and deploying contracts on your shadow fork

Free trial

Use Shadow to get custom data for any contract in minutes with a free trial account.

To get started, just go to and login with a Google account.

See our guide to get set up with custom onchain data in minutes.

Demo

Paid plans

Here are Shadow's . If you'd like to set up a custom pricing plan, please fill out the form below or reach out to .

Get in touch

Resources

By the end of this tutorial, you will:

1. Add a custom shadow event by editing the WETH contract

2. Test your changes

3. Deploy your changes to your shadow fork

4. Make a JSON-RPC request to retrieve your custom shadow logs

Step 0 – Login to Shadow

Login to with a Google account.

You should see a screen like the one shown below. The string after project_id= in the URL is your unique Shadow project ID (e.g. fc6ee70a-6f14-4a99-81db-6bd96226b95f).

Step 1 – Add a custom shadow event

Navigate to the WETH token contract page by pasting the contract address into the search bar 0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2

Click the “Open in editor” button on the top right to open the in-browser IDE.

At the top of the contract on L27, define a new event schema called HelloWorld:

Then, add a line to emit a HelloWorld log at L74 with a custom message every time WETH is transferred:

After you’re done with your changes, hit the “Compile” button on the top right.

Step 2 – Test your changes

Test run your changes by simulating a transaction, and confirming that the transaction has logged your new shadow event.

Click the “Test Run” button on the top right, paste in the following transaction hash, and hit “Run”: 0x2ce5687567574b47c9406df5b65c1d9dd0d94bdcdde7109810b642c35599774b.

You should see your new HelloWorld shadow event in the right hand panel!

Step 3 – Deploy your shadow fork

Now that you’ve tested your shadow changes, you’re ready to deploy the shadow contract onto your shadow fork. Just hit the “Deploy” button on the top right corner.

Step 4 – Make a JSON-RPC request

Now that you've deployed your shadow changes, you're ready to request the data from your shadow fork's RPC. Here's how you would make an eth_getLogs request from your terminal.

That's it! You have successfully generated custom data on your own shadow fork 🌗

Every time you deploy new changes to your shadow fork, a new version will be automatically created with an auto-incrementing version number. This ensures that each version of your shadow fork is immutable, preserving the state and configuration at the time of deployment.

These immutable versions can be reliably used for exports, database syncs, or JSON-RPC calls.

Immutable versioning allows you to safely deploy changes to your shadow fork and test them without disrupting previously running jobs.

For example, if you set up a recurring export on your shadow fork at version v2, that recurring export will always be run using the shadow changes that were deployed at v2.

If you deploy new changes to your shadow fork:

For a 5 minute guide on how to modify a simple contract, see our .

In this more advanced section, we'll go through how to modify contracts on your shadow fork that use design patterns that are commonly found in the wild:

1. that delegate call to implementation contracts.

2. to get token prices in USD, without using offchain price APIs.

All API requests to Shadow are authenticated via API keys. This includes Shadow RPC requests, as well as api.shadow.xyz endpoints. You can create a new API key by going to your project’s settings page: .

Add your API key to the request header to authorize your Shadow API requests. This header must be included in every request.

X-SHADOW-API-KEY: <YOUR-API-KEY>

For example, this is how you’d authenticate a call to your Shadow RPC:

In this example, we’ll be making interdependent changes to Blur’s Blend and BlurPool contracts.

We’ll be making changes so that the BlurPool contract will emit a new shadow event TransferForLien every time some ETH in the pool gets transferred due to an action taken on a given lien (e.g. borrow, repay, refinance). The TransferForLien shadow event will include metadata about the lien.

Step 1 – Edit the BlurPool contract

Open the example on the Shadow Playground: , which will open the playground editor at the BlurPool contract.

In BlurPool.sol on L115, we’ve introduced a new function called transferFromForLien:

You can see the schema of the TransferForLien event in the IBlurPool.sol file:

Step 2 – Deploy the BlurPool changes to your fork (optional)

If you want to apply these changes on your shadow fork, click the button on the top right corner that says “Apply to your shadow fork”. This will take you to the editor for your shadow fork, where you can deploy the changes by hitting “Compile > Deploy”.

Step 3 – Edit the Blend contract

Go to the following URL to open the Blend contract in the Shadow Playground for this example.:

In Blend.sol, you’ll see that we updated all BlurPool.transferFrom() call sites to call the new transferFromForLien function we added earlier.

Step 4 – Test run your changes

Click “Compile” > “Test Run” in the top right corner, and paste in the following transaction hashes to see these shadow changes in action:

• Borrow:

• Repay:

• Refinance:

You should see your custom TransferForLien event in the output for each transaction!

Step 5 – Deploy the Blend changes to your fork (optional)

If you want to apply these changes on your shadow fork, click the button on the top right corner that says “Apply to your shadow fork”.

This will take you to the editor for your shadow fork, where you can deploy the changes by hitting “Compile > Deploy”.

You can use Shadow to tackle several types of onchain data problems:

Faster and richer indexing

Teams can cut engineering time, get rid of adapters, and speed up indexing by writing shadow events from the contracts they care about. Shadow RPCs support the standard read endpoints, and are backwards compatible with existing web3 tooling like subgraphs.

Deeper research and analysis

Events often don't have all the data you will need. This can stem from a variety of reasons:

• Events are missing entirely

• Existing events don't emit information you need

• Existing events emit inaccurate information

Analysts and researchers can take matters into their own hands and write shadow events into any contract. Write a custom event, quickly backfill its data for past transactions across a block range, and export it to your data tool of choice.

Save gas and contract size

Smart contract developers can save their users up to 12% in gas on every transaction by decoupling data access from business logic, and moving non-critical event logs and view functions to a shadow fork. This has the added benefit of reducing the code size of contracts deployed on mainnet, which has a limit of 24KB (the Spurious Dragon Limit).

Reducing events on the publicly deployed versions of your contracts – and deploying them on a shadow fork instead – can provide significant optimizations to your contracts while flexibly serving all of the data needs for your project.

Complete observability

Developers and auditors can use Shadow to see inside the entire stack of any contract, and log everything they might want privately and gaslessly on a shadow fork. This enables fully comprehensive semantic logging to understand the inner details of any contract or transaction.

Data ergonomics

Shadow provides powerful and flexible data middleware that simplifies any indexing you may need to do downstream of events.

Shadow enables you to easily:

• Materialize SQL-ready enriched tables

• Make eth_call data event-driven instead of hammering RPCs

• Unify data models for similar user actions across different protocols

Recurring exports allow you to export data to a destination of your choice on a recurring basis.

Steps

1. Choose your data: Export shadow and/or mainnet data (coming soon), and specify which contracts and events you want to export.

2. Choose your destination: Export your data to Dune Analytics, GCS, or S3 (coming soon).

3. Choose your settings: Specify the starting block you want to export for, the file format (CSV or Parquet), and how to partition the data (daily, hourly, or no partitioning).

The GCS / S3 integration is available for One-off Exports and Recurring Exports. It's ideal for teams who need historical blockchain data for analytics or research use cases that don’t require real-time data freshness. Data exports to GCS / S3 offer the highest flexibility, allowing you to upload data into tools that you already use such as Snowflake, Metabase, etc.

When you set up a data export to GCS / S3, you’ll be able to specify:

1. Which contracts and events to export

2. The file format (CSV or Parquet)

3. How often you want the data delivered to your bucket (daily, hourly, every 15 min)

4. How to partition the data (daily, hourly, or no partitioning)

See the or pages for additional information.

The Dune integration is available for one-off exports and recurring exports. It's ideal for teams who want to make their shadow events accessible to the community, or for teams who primarily use Dune as a tool for data analysis.

When you set up a data export to Dune, you’ll be able to specify:

1. Which contracts and events to export

2. How often to upload the data to Dune (daily, hourly, every 15 min)

You’ll also need to provide a Dune API key, which you can create on Dune following these . Make sure that your Dune plan allows you to upload the amount of data required for your data export.

See the or pages for additional information.

One-off exports allow you to export data to a destination of your choice for a specified block range.

Steps

1. Choose your data: Export shadow and/or mainnet data (coming soon), and specify which contracts and events you want to export.

2. Choose your destination: Export your data to Dune, GCS, or S3 (coming soon).

3. Choose your settings: Specify the block range you want to export over, the file format (CSV or Parquet), and how to partition the data (daily, hourly, or no partitioning).

What is a factory contract?

Factory contracts are a widely used pattern across DeFi in DEXs, lending pools, and yield vaults.

A factory is a contract that can deploy multiple instances of the same contract — we call these child contracts.

Generally, child contracts share the same functionality, but are typically initialized with different constructor arguments, which may alter the child contract’s runtime bytecode. Factory contracts are widely used across many protocols, including Uniswap, Compound, and Pendle – for example, there are 300K+ Uniswap V2 pools deployed by one factory contract.

Shadowing each pool contract individually would be a highly tedious and impractical process. Shadow provides powerful features that allow you to make one set of changes to a factory contract and link all of its child contracts to those changes.

Product guide

Identifying a factory contract

Shadow automatically detects if a contract is a factory contract, or a child contract deployed by a factory contract. On the contract page, you will:

1. See indicators that tell you:

   • That this is a factory contract

   • If the factory contract is current linked with all of its child contracts

Linking a factory contract to its child contracts

If you already have deployed changes to a factory contracts on your shadow fork, you can simply click the Link Children button from the factory contract page to link it with all of its children.

Once linked, any changes deployed to the factory contract will propagate to its child contracts; this includes all existing child contracts, as well as any child contracts deployed by the factory in the future.

You can also link a factory contract to its child contracts from the Editor, while deploying changes.

Demo

Overview

Your Shadow Catalog allows you to keep track of the contracts and events that you're interested in.

Contract Catalog

To add a contract to your Catalog, click the + icon next to the contract name. You can add any contract to your Catalog, even if you haven't made any edits to the original deployed version.

Adding or removing a contract to your Catalog has no bearing on the modifications you may have made on that contract; it's just a way for you to keep track of things in the web app. Your shadow fork will continue to execute transactions on the version of the contract that was last deployed.

Once you've added a contract to your Catalog, you'll be able to see it in your Contract Catalog tab.

Event Catalog

To add an event to your Catalog, click the + icon next to the event name in the Event Schemas tab of a contract page. You can add any event to your Catalog, even if it's not a custom shadow event.

Adding or removing an event to your Catalog has no bearing on the modifications you may have made to the event; it's just a way for you to keep track of things in the web app. Your shadow fork will continue to execute transactions on the version of the event that was last deployed.

Database syncs allow you to stream data to your own database in realtime.

Data will appear within seconds of a transaction on mainnet landing, and Shadow handles re-orgs for you automatically by replacing rows.

Steps

1. Choose your data: Export shadow and/or mainnet data (coming soon), and specify which contracts and events you want to export.

2. Choose your destination: You can send your data to your own Postgres database.

3. Choose your settings: You’ll be prompted to set a start block for the recurring export if you’d like to export historical data.

The Postgres integration is available for . It's ideal for teams who need real-time, reorg-aware, data for indexing or analytics.

With data streamed directly into your Postgres instance, you can build REST or GraphQL APIs tailored to your specific use cases. This eliminates the overhead required of managing real-time data pipelines, allowing you to focus on core application development.

To set up a database sync, you’ll need to provide:

• Host address

Shadow currently supports the following chains and networks.

If you'd like us to support a chain that isn't listed below, let us know at [email protected].

What is a proxy contract?

A proxy contract in Ethereum is a smart contract that acts as an intermediary or wrapper for another contract, enabling features like upgradeability and interaction redirection.

Proxy contracts make delegate calls to a designated implementation contract, which contains the actual smart contract logic. This setup allows the logic (implementation contract) to be updated without changing the proxy contract's address, preserving the contract's state and address on the blockchain. Proxy contracts are a core component in creating upgradeable smart contracts.

Do I write shadow events on the proxy or implementation?

If you want to get custom data for a contract that uses a proxy pattern, you'll need to modify the code of the implementation contract, because that's where the actual contract logic resides.

When transactions happen, events will get emitted by the proxy contract, while the event declarations and event emit logic reside in the code of the implementation contract. This is the same way other platforms that index smart contract data handle proxy patterns.

Tips

Add both the proxy and implementation contracts in Shadow

You will want to add both the proxy contract and implementation contract to your , because you will need to:

• Edit the logic and add shadow events to the implementation contract

• View the event emits on the proxy contract

How do I tell if a contract is a proxy contract?

On the Etherscan contract page, click on the Contract subtab. If you see the Read as Proxy tab, that means that this contract is a proxy contract.

Here's the Aave V3 Pool contract, which uses a proxy pattern, as an example.

How do I find a proxy contract's implementation contract?

Click on the Read as Proxy subtab. The current implementation contract address will be listed.

What do I do if a proxy has multiple past implementation contracts?

If you want to get shadow event data across a time period where the proxy contract has changed the implementation contract that it delegates its logic to, you will need to write separate shadow event logic on each implementation contract that was used during that time period.

This is because the logic across the previous implementation contracts could be different. Shadow will show your custom data from all implementation contracts on the proxy contract.

A proxy contract is a smart contract that forwards transactions to another implementation contract containing the business logic. The proxy acts as an intermediary, allowing the implementation to be updated without changing the contract address users interact with.

For this example, we will be shadowing USDC to emit a custom event on every transfer. The custom event will be called TransferWithMetadata, and will contain additional metadata about the transfer, including the before and after balances, and the percent change.

USDC follows a proxy pattern with the following addresses:

• Proxy:

Without Shadow, augmenting blockchain data with point-in-time accurate prices requires complex data pipelines and integrations with third-party price feed APIs. Shadow drastically simplifies this by allowing you to leverage existing oracle contracts, in <100 lines of code.

In this example, we’ll be updating the 1inch Aggregator contract to emit a shadow event called OrderFilledDetails that contains additional metadata of a trade, including the point-in-time USD price of the trade. We’ll be fetching the point-in-time prices via a Chainlink oracle.

Step 1 – Edit the contract

Open the example on the Shadow Playground:

Shadow RPCs support the read methods required by most subgraph services.

You can run a local subgraph pointed to a Shadow RPC today, and we're actively working on integrations with hosted subgraph providers.

Why use Shadow with subgraphs?

You can significantly reduce the time to resync your subgraph by moving pieces of indexing logic into custom shadow events. For example, if your subgraph currently:

1. to help you write shadow events

2. to give GPT context on Shadow and provide ready-to-use code

3. extension to pull down contract code by pasting in an Etherscan link

Shadow's test run feature allows you to preview shadow events for confirmed transactions, before actually deploying changes to your shadow fork.

This makes it easier to:

• Iterate on your shadow contract changes

• Verify that you're getting the data you expect

• Inspect particularly complex transactions

Steps

1. Open editor mode for a contract and make changes

2. Compile your shadow contract

3. Click "test run" and paste in a transaction hash