You asked and we delivered. Our clients have asked us for more control of their data on the Empower Engine platform, and we’ve made many improvements around that this spring. The biggest improvement is allowing admin users to decide what data gets updated in our overnight processes.

Data directors can now set up their own lists of voters (e.g. 1s & 2s, persuasion targets, GOTV targets, etc.) from tables in a data warehouse. Campaigns without data directors can still share their saved lists to us in a special “Lists to Map” folder in Votebuilder.

We’ve also empowered our admin users to create penetration datasets and early vote datasets from their lists of voters. You can now do it all on your own.

Automatically updating datasets made from your VAN data has always been a feature of Empower Engine. But in the past, our staff had to set up each dataset individually for each client. Now we’ve given our users control.

Admin users will have a page like this showing what is updated in their overnight job. In this case, our imaginary Gryffindor client has one dataset showing their supporters and another showing their GOTV targets. Within their list of targets, they’re also watching the effectiveness of their texting program on maps showing which targets have been texted since their field program launched. They can also see which of their targets have/have not yet voted.

Creating new lists of voters (Data Staff Only)

Let’s say that you want to see add a new list of GOTV targets on Empower Engine. To do so, just create a table in a shared space on your data warehouse (e.g. the DNC’s BigQuery) and then fill out the simple form on your Overnight Job page. You can change the set of voters as often as you want since we’ll always get the most recent values when we update your data. E.g. A table of people with a specific activist code should constantly grow in size.

Here’s what we need to know:

How to name your list of voters.
Where to find your list of voters. This will be the “project.dataset.table” path to the table on the DNC’s BigQuery or an equivalent path in another data warehouse.
Which voter id column the table contains (e.g. VanID, PersonID, etc.).
How to categorize the dataset in Empower Engine (e.g. Field, Fundraising, GOTV, etc.).
Who should have access to the dataset in Empower Engine (only relevant for multi-committee users).
What geographic levels of detail at which we should aggregate the data.

Just fill out the form, and your new dataset is good to go.

Right now, our system is fully implemented and in active use with the DNC’s BigQuery. It was designed to be general-purpose, so we should be able to easily work with Civis or any other data warehouse system with minimal tweaks.

Creating Penetration Datasets (Any Admin User)

Setting up a penetration dataset to see your progress in the field is easy. Whether you’ve set up a list of voters using the process above, or whether we’re pulling a “Lists to Map” saved list for you, you can create datasets showing the reach of your field work.

In the example below, our Gryffindor admin user has a list of Gryffindor GOTV targets, and she is now creating a penetration dataset tracking who has been texted since May 1st.

Using our system, you can see the work done on one VAN committee or combined across multiple committees. This is particularly important if you’re working for a caucus or if your coordinated campaign has multiple VAN committees.

If our admin user wanted to set up phone penetration datasets (or doors datasets in a cycle where they’re relevant), she could do that with this same form. In that case, she’d automatically get two penetration datasets -- one showing all attempts and one showing just contacts. Users can also create penetration datasets that combine all types of field work; then texts are only included in the attempts dataset.

Creating Absentee/Early Vote Datasets (Any Admin User)

Given the COVID-19 pandemic, absentee ballots (and to a lesser extent in-person early voting) is even more of a focus than usual. With that in mind, it’s important to note that you can use this system to see what portion of the voters in your lists have requested a ballot, have absentee voted, have early voted, or which have already voted (early & absentee combined).

Which data is available in your state will of course determine which of these options are relevant. (We rely on the contacts_absentees table from VAN.)

Just as we easily set up a penetration dataset above, we can set up early vote datasets. For example, selecting the setting shown below will create a dataset to show the proportion of targets in the (yet to be selected) list who have already voted, whether they voted absentee or used in-person early voting. It will create a second dataset to show the density of targets who have not yet voted -- your current GOTV targets.

All Updating Data

If a list of voters is no longer relevant, you can remove it and stop all the datasets associated with it from updating anymore. They will still be available for historic comparisons. E.g. You can see maps of what was done for the primary as you get ready for the general. You can also go to the details page of any specific dataset to rename it or to stop it from updating.

Whenever you create a new dataset -- a list of voters, a penetration dataset, or an early vote dataset -- it will be created immediately. It won’t have values until the next time your overnight update runs (generally daily or weekly), so you’ll need to wait before creating Power Maps from the new dataset. If your job won’t be run for days after you’ve added a new dataset, or if you need to look at the dataset right away for any reason, just shoot us an email. We’re always happy to run any overnight job during the day as a one-off.

Visualizing data that updates has always been a central feature of our platform. As we gear up for 2020, Empower Engine has taken a huge step to give you greater control.

Update: You can now set up your own nightly updating data!