Adobe AEP Guide: Datastreams

Let’s get straight to the point. Datastreams make sure data gets from your source (like a website) to your destination (like AEP). A datastream can also transform your data from ugly to pretty (or bad to good) with the Edit Mapping feature (AKA “Data Prep”). Additionally, the Edit Mapping section (Data Prep) lets you pass in something like a data layer and parse that stuff into your schema. So if you don’t want to use the Web SDK extension in Launch, you can just pass that data straight into your datastream and parse it there. Maybe that’s all you need to know. If that’s it – thanks for reading! It’s been great.

Wait, you want more context? Heh, okay well let me turn to Adobe’s documentation.

Reading that is an uphill battle. Like I know these words individually… but I feel like this opening paragraph assumes you have a bunch more context than the average AEP user. When you start talking about server-side configuration and sending data to an AEP Edge Network with some client named edgeDomain with some variable named edgeConfigId I’m like “wait wait wait, can we back up about 200 steps here?”

The next section goes right into talking about sensitive data handling… and I’m still DM’ing people asking what half of the stuff in the first paragraph means. Maybe I don’t REALLY know what a Datastream is, but you’ve gotta have one! Let’s take this journey together and figure this out so my small brain can comprehend these words in the context of AEP.

How do datastreams work?

What is this thing? I think know what it is…? Like I’ve created a bunch of them and clicked the buttons that said “send my data there” – but is that all there is to it? Obviously that’s not the case. I’m going to try to keep this in the realm of stuff you probably need to know to understand it without going into so much detail you throw up. Let’s start with that overview paragraph. If I’m brand new and reading this, the bit I completely understand (the last sentence) suggests that this operates like a tag management system. But this isn’t the same as Adobe Launch (or Adobe Data Collection Tags). This also isn’t the same as Adobe’s Event Forwarding. Each of these technically do the same kind of thing – make changes without messing with code on your website.

A Datastream is like a pipe (and apparently a cab driver if you prefer that metaphor). It carries stuff from one place to another. Since you read past the first sentence, it’s worth noting that it also does more stuff. It hooks into all of Adobe’s services like all of the tools in the Experience Cloud – and, of course, AEP (which means it hooks into AJO, CJA, etc). There’s also Event Forwarding, which is the tool that sends your data to other places – like Meta or LinkedIn.

The above image lists the services you can add to your datastream. I’ll refer to this toward the end of this article as the “service config”.

Where do datastreams sit in the AEP workflow?

So we’ve talked about what datastreams do – and you might remember that it sits between the source and the destination. What does that mean, though? The above image shows how everything works together (for 1 use case). My labels are probably imperfect here. For instance, a data layer should live somewhere in there or you might not be using Launch. Hell, you might not even be sending data to a dataset (like if you wanted to send data to Event Forwarding)! This is probably the most common workflow, though. Pretend there are arrows after the dataset to an application like CJA.

A single datastream could also be used to service multiple sources:

Like the image suggests, if you have a website and a mobile app you might want to send data to the same dataset. In this case, you might use the same datastream (for consistency).

When would I use different datastreams?

Let’s say you have a mobile app and a website each passing data into the same dataset. I might want to use a separate datastream to use data prep to massage the data so it is consistent between both data sources. Maybe you’re sending the XDM schema from the website and a JSON object on your mobile app. In a perfect world, you would use 1 datastream for both sources, though.

When would I use multiple datasets for one datastream?

This is a bit of an outlier, but I might use multiple datasets if I’m doing data transformations or mapping for one set and collecting raw values in the other (this would be done in data prep). Another use case might be if you have one dataset that’s configured for profile and another that isn’t. I’m not sure I have useful justification for that, but there ya have it.

What the heck is “data prep”?

If you’ve been poking around AEP, you’ve probably heard people use the term “data prep” (I even mentioned it a few times in the last section). It even has its own documentation page! “Data prep” is like Adobe Analytics processing rules… but for datastreams (and also for datasets). The technical name for it in the context of datastreams is “Data Prep for Data Collection”. It does stuff to the data before it’s sent off to its destination. So I guess it’s like processing rules on steroids. NOTE: the word “Data Prep” is NOT listed in the interface. In the datastreams interface, it’s a button that says “Edit Mapping”. Read this again – when people say “Data Prep”, there is nothing in the interface that says “Data Prep”. When someone says that, they’re talking about the Edit Mapping section in your datastream.

There could be an entire article written about mappings (which honestly should be used sparingly), so I’m going to cut this section a little short.

Final Thoughts

Let’s come full circle to that highlighted paragraph from the beginning:

I hope datastreams make more sense. Now if you’ve been paying attention, you might be wondering where you should transform your data. Should you transform your data in your data layer, Adobe Launch (Data Collection Tags), or your datastream? In a perfect world: your data layer. The more places you’re transforming your data, the harder your data is to debug. Can’t get development support? Then I’d transform in Launch. Can’t do that? Well then I guess use the mapping.

That said, there is a time and place for data transformation to happen in datastreams. One specific use case is with the Geolocation, Network, and Device Lookups.

This is baked-in stuff that automatically pops into places like CJA (in place of your User Agent for Devices) and whose variables can be used to send data elsewhere. Here’s a little video I did on the Device Lookup:

Datastreams are necessary and exist to transport your data. The big question is whether it exists to transform your data. I’m not sold on that one, but there will be a time/place for it and the functionality is critical. In a world of theatrical legislation, new platforms, and an increasing dependence on technical aptitude – keep it simple. Simple is hard. Simple means you push back and do what’s best long-term. Take extra care to strategically move our industry’s maturity forward instead of letting things happen to us. Complacency is a consultant’s best friend.

Keep an eye out for my next guide on Datasets!