Introducing AEM as a Cloud Service – Diagrams & Explainer

For people like me who have spent the last 10 years of their life wrangling AEM (or CQ5) infrastructure, AEM as a Cloud Service (which I hope Adobe gives a more google-friendly name) is the single biggest shift to the way all of us imagine deploying AEM, and the most massive shift to the way operations-focused folks view our own career future.

It’s a big freaking deal, and you should understand what it is and what it potentially means.

What is “AEM as a Cloud Service?”

AEM as a Cloud Service is a yet-to-be-generally-released service offering from Adobe, which is the next evolution of a managed AEM offering after Adobe Managed Services. Just like it sounds, instead of creating a managed AEM infrastructure with individual servers that you provision, license and maintain, AEM as a Cloud Service further abstracts the hardware layer of AEM, taking care of scaling AEM vertically and horizontally for you as appropriate, and presenting you only with the AEM endpoints you need to author & publish content, and interacting with the code on AEM exclusively through Adobe’s Cloud Manager CI solution.

It’s easier to explain visually. Here is what the current Adobe Managed Services solution looks like.

When a basic AMS environment is provisioned, it consists of classic TarMK AEM Author & Publish instances and AEM dispatcher instances, pre-provisioned with as many as appropriate for that environment based on the contract you’ve got with Adobe. Functionally and technically, there is no real difference between an AEM environment provisioned in Adobe Managed Services, and an AEM environment you might create by yourself at your local datacenter or in AWS/Azure, with the only major differences being that in AMS, authentication is provided by Adobe IMS, deployment is done using Cloud Manager, and you don’t have direct access to install your own APM tooling (like Dynatrace/etc) and you don’t have console access to the boxes.

AEM as a Cloud Service, however, looks like this:

There are a few very key, very significant changes in this architecture. They are:

AEM Structured Content and AEM Code are Separate

The architecture of AEM as a Cloud Service comes from project that’s been running for almost two years at Adobe – an initiative to be able to make AEM container-friendly called “Project Skyline”, by finally separating out blobs & structured content from the AEM factory codebase, and to keep that in a separate “Content Repository Service”. This way, one can have standardized AEM Publisher and Author Modules that can be vertically and horizontally scaled by a separate Orchestration Service, entirely separate from one’s AEM content, binaries and custom code.

This allows Adobe to much more-easily upgrade servers without impacting code, to add and remove capacity from a pool of Author or Publish servers, and to replace unhealthy instances without causing interruption.

This video from Adapt.to this past fall is 48 mins long, but is the best technical description I’ve found on how this works, and is a must-watch if you’re interested in how all these services play together:

This is an architecture that (presuming it works) would solve AEM’s biggest headache, which is that it is extremely cumbersome to scale, given that every instance has to have a full copy of all code, content, and the full configured state of that instance. AEM Publishers and Authors are commonly sized in the TERABYTES, and so quickly cloning, adding and removing another 1TB volume from a cluster is never going to be an instant proposition no matter how good your automation is. This finally promises to solve this.

Update: Some links from Adapt.to were given to me from Joerg Hoh, which you should definitely peruse if you’re interested in the technical background for how this composite node store works. This presentation goes over the Oak Composite Node Store and its use in a containerized AEM Author Cluster, and this is a deep-dive on cloud-native AEM deployments using Kubernetes.

The Rise of “Version-Less” AEM

With the launch of AEM as a Cloud Service, Adobe is able to move to versionless, fully continual release cycle as opposed to the annual version / quarterly service pack release cadence that has been the standard up until now. Given that Adobe can now completely control the release process (with Cloud Manager being the only way to get code into an AEM Cloud Service system), Adobe can then also release features much more rapidly and transparently, essentially (hopefully) doing away with the “upgrade projects” that have really defined life in the AEM/CQ world since the beginning.

How exactly this will pan out in the real world is yet to be seen, but Adobe will soon make available local AEM instances that are version-updated in lockstep with the current release, which developers can use for local software development. Then, one just provisions as many cloud service AEM environments as needed to run your development pipeline, and theoretically everything should work just as it would in a normal AMS environment.

Asset Microservices for Serverless Asset Processing

If you’ve read through what I went through to attempt to get AEM workflow offloading to work properly, for a large AEM Assets installation (or if you’ve been through similar headaches yourself) you’d know that a major limitation of AEM has classically been that the AEM Assets Author is a major bottleneck if your company has a lot of digital assets. Expecting a single compute instance to simultaneously provide the UI for a robust CMS while also transcoding large videos, extracting metadata and generating renditions from large Photoshop files, and re-encoding multiple sizes of PDF files, is just a recipe for Ops Engineer headaches and laggy UX.

Adobe has finally solved this by creating an Assets Microservice powered by Adobe I/O.

Now, as opposed to the way it currently works – with the AEM Author receiving uploaded files and kicking off & executing rendition and transcoding workflows itself, digital assets uploaded to an AEM Cloud Service installation will instead be uploaded directly to cloud binary storage. The Adobe IO runtime is then leveraged to kick off whatever configured rendition and transcoding jobs that you have configured, entirely outside of the AEM application, complete with any Adobe Sensei-powered imaging jobs like Smart Tagging, Smart Crop, etc. Dynamic Media is also enabled by default on AEM Cloud Service instances.

Then, all AEM has to do is receive, store and search upon the extracted asset metadata, as opposed to having to transcode massive video files itself. The AEM Desktop App is now also optimized for this workflow, which means that one can finally really expect people to USE the AEM Desktop app, instead of being terrified that they’ll find out about it and take your Author instance down with massive unbridled uploads.

Is there Still a Case for Non-Adobe AEM Hosting?

If you ask Adobe Sales, no. No there’s not. For the last few years there have been indirect replies to questions I’ve asked Adobe on the subject, on how Adobe Partners like me and companies I work for fit in to the ecosystem. Their FY20 goals announced at the Adobe WWSC this fall, however, are much more clear:

“Move all On-Premise Customers to AEM as a Cloud Service.”

That’s pretty clear. Talking with other Adobe engineers at Summit last year, though, made it clear that Adobe Engineering does acknowledge there’s going to continue to be a case for standalone AEM hosting for much of the foreseeable future.

Which is good for Adobe Partners such as my own company who provide extremely robust, extremely flexible, extremely evolved infrastructure solutions for AEM.

After getting to play with some sandbox editions of AEM as a Cloud Service, many of the points noted in this much fuller article comparing Adobe Managed Services to Standalone Hosting, still stand with the AEM Cloud Service, namely:

Infrastructure Choice

There are still myriad reasons why one might want to have a choice of infrastructure for AEM. If you are a financial services company, you may have strict controls about where your content can go physically, which may preclude you from running your corporate site on such a cloud service. You may have an internal-only site that is inappropriate or from a networking perspective is precluded from being able to migrate to such a cloud service.

AEM is used for probably way more things than it even should be used for, and as a result the hardware and infrastructure requirements imposed on its deployment are just as varied. There will, for the conceivable future, always be requirements that cannot be adequately met by a fully shared-tenancy cloud service solution.

Integrating Other Services and Non-AEM Infrastructure

AEM is seldomly deployed on its own. It frequently needs to connect to corporate authentication systems, frequently has external databases, application servers, or search clusters which are used in conjunction with AEM’s powerful experience management features to deliver the overall solution.

This is one area where Adobe’s boffins will have to work hard to provide solutions to acceptably run AEM as a Cloud Service alongside the bevvy of other infrastructure which commonly comes along for the ride.

Deployment Pipeline Flexibility and Capability

The only Adobe can enable the type of version-less architecture that AEM as a Cloud Service has is to tightly control and standardize the deployment process, so that Cloud Manager and an Adobe-managed git account is the only way to get code in and out of the system.

However, this is one area where companies have WIDELY different needs and processes to support, in terms of how development gets done, how code is deployed, and how major releases and parallel projects are managed. Many companies have a need to have multiple individual git repos for various projects and deployment artifacts, and in many cases use highly-evolved Jenkins, CircleCI, Bamboo or other continuous integration solutions in conjunction with a host of other testing and quality tools to provide a reliable pipeline.

Moving to AEM as a Cloud Service would mean a complete dismantling of any existing process you have, and moving to a highly-controlled and simplified development pipeline supplied by Cloud Manager. This may not impact smaller shops, and for some companies may actually be a huge plus. But for other companies with very evolved deployment strategies may make Adobe Managed Services a complete non-starter.

Dev/Ops Observability Tooling (Log Aggregation / APM)

As this is a fully-managed Cloud Service, one of course would not have lower-level system or console access to the underlying gear running AEM. But this also extends to logging, and at the current time the only way to get access to log files from AEM as a Cloud Service is through a download link in Cloud Manager where you can make a download of your current AEM request / access / error / dispatcher logs.

This means no Splunk, no log aggregator, no tailing of log files, and no searching. This may be an area which Adobe chooses to improve upon in the future, but for the time being it will make troubleshooting very cumbersome.

It also means that you will not have the ability to use New Relic or Dynatrace or AppDynamics or Datadog or whatever your APM tool of choice is on AEM as a Cloud Service. It’s possible that in the future Adobe may make their own internal New Relic tooling available for customers to view, but at the time being this is not something you can do anything about. So, troubleshooting slow AEM environments will be a bit of a chore until this issue is traversed.

AEM Screens / Forms Support

Minor, but at the time being Adobe Forms and AEM Screens are not supported in AEM as a Cloud Service. They are indeed working on this, so this may be supported by the time Adobe Summit rolls around.

Summary, and Note for Adobe Folks Reading This

As someone who’s made a living supporting and architecting AEM infrastructure for the last decade, nothing has really changed my career path quite as certainly as the announcement of AEM as a Cloud Service. It certainly paints a pretty sure picture of what the future world of AEM is meant to look like, and also serves to redefine the role that AEM consultants like me will likely play in the future.

Now, there are a lot of questions still about how this will be rolled out, many of which I’m sure will be answered as we approach the Adobe Summit this April. Namely:

• How will AEM as a Cloud Service be priced? AEM has always traditionally used a per-instance-in-prod pricing model, but with a service that’s always scaling on its own, with no observability into how its performing on the back-end, how will this be priced? You could price it by page view, but then there’d be no penalty for shops that don’t optimize their code and have slow queries and zero caching and thus have to scale up to 50 publishers to support their load. How will this roll out?
• What of On-Premise customers? For big customers which huge AEM footprints (like companies such as Autodesk and Cisco who very definitely can’t fit into Adobe Managed Services), what will be the upgrade and supportability path for them? Have a look at this slide from Autodesk’s presentation at the AEM Evolve conference this past August – they are doing some AMAZING things with AEM which certainly wouldn’t fit the mold of a one-size-fits-all cloud service, so I’m very interested in Adobe’s enablement strategy for enterprise customers like that.

I hope to continue to find out more as the months drag on, but for sure this is an exciting to be around in this space!