A Flight to AG Carinae

This blog post is one in a series marking the 31^st anniversary of the launch of the Hubble Space Telescope. For more information and resources regarding Hubble’s 31^st anniversary, please visit hubblesite.

The Hubble Space Telescope recently celebrated its 31^st year looking deep into the cosmos to bring us views that continue to awe and inspire us. The image of this year’s target, AG Carinae or “AG Car” for short, is a fascinating display that happens to look somewhat like cotton candy. In truth, it showcases the effects of living life as a very massive star that doesn’t quite know what size it wants to be. As the image data was gathered and processed, the visualization team carefully studied the details of the gas-and-dust shell around the star, and eventually decided to bring this 2D image to 3D life.

To learn more about AG Car and the image processing process check out this post by Joseph DePasquale and Alyssa Pagan.

AG Car—A Luminous Blue Variable Star

Before attempting any sort of visualization, one has to understand the basic astronomy and physics of the situation.

Researchers have written dozens of papers on AG Car and similar stars, and we spend many hours skimming, reading, examining images and figures, and absorbing the important points. When possible, we consult researchers to check our understanding and glean further insight. Special thanks to Antonella Nota and Kerstin Weis for their knowledge and advice on this project.

AG Car is classified as a “luminous blue variable” star. “Luminous” means the star is intrinsically very bright, about a million times brighter than our Sun. It is a one of the brightest stars in the entire Milky Way Galaxy, and, since there are about 200 billion stars in our galaxy, that’s really saying something. “Blue” is the dominant color of the star in visible light, while “variable” means that the brightness of the star changes over time.

The variability of AG Car is fundamental to understanding the nebula that surrounds it. The graph above shows changes in brightness of about a factor of 60 in recent times. These changes can be caused by the star swelling and shrinking in size, as well as by the star casting off material from its outer layers. Luminous blue variables are known to have huge outbursts of material. That cast-off material from an eruption about 10,000 years ago forms the bulk of the gas in the nebula.

In addition, the star emits a strong stellar wind of charged particles. This outflow, moving at about a million kilometers per hour, clears a cavity around the star and pushes the expelled gas out into a shell. The dust in the shell shows strong “fingers” that point back to the central star. These pillar-shaped features are created as the wind flows through the material.

The critical idea for a visualization is that the nebula should be a shell of material that is moving away from AG Car.

A Super Space Peanut—Deducing Shape from Motion

The basic shape of the nebula around AG Car is roughly circular on the sky. The bright star in the center is surrounded by ionized gas and dust. Looking deeper into the gas emission, one sees an extension (toward the top in the right image above) that one researcher dubbed the “cone.” This appearance is a 2D projection of a 3D shape.

The 3D shape can be untangled by looking at the motion of the gas, something called Doppler shift. Gas that is moving away will have its light stretched to longer wavelengths, which is called redshift. When moving toward the observer, the light will shrink to shorter wavelengths, called blueshift. Astronomers use a filter to isolate the emission of a spectral line, like Nitrogen in the images above. Then, knowing the standard wavelength of that emission line, they can take a spectrum to see which parts of the nebula are blueshifted (approaching) or redshifted (receding).

As noted above, we expect the nebula to be an expanding spherical shell of gas. That structure would show an ellipse in the spectrum. AG Car, however, shows two ellipses in its spectrum. One ellipse is an approaching, blueshifted shell that is outlined in blue in the figure above. The other ellipse is a receding, redshifted shell outlined in red. AG Car has what astronomers call bipolar lobes, or roughly the shape of a peanut. Our view is pretty much down the axis connecting those lobes, with just a small amount of rotation. The two bubbles mostly overlie each other, which gives the mostly circular appearance.

In addition, the spectrum of the cone indicates that this gas is flowing away at higher speeds than the the gas of the receding shell. The gas in this region is a blowout, likely created by one of AG Car’s eruptions long ago. It is extended out behind the rest of the nebula, like a cape flowing behind a superhero.

Combing the two-lobed shape with a cape trailing behind it, one can call the shape a “Super Space Peanut.” While that may sound silly, such inventive imagery helps to communicate properly between scientists and artists.

A Visualization Takes Shape

After studying the object and determining a scientifically reasonable shape, we discuss and identify the appropriate visualization methods. Here, we chose a decoupage method, which layers numerous 2D images to produce a 3D effect. The layers are sculpted to enhance and emphasize structures as the camera changes perspective. A primary benefit of this sculpted decoupage technique is that the initial frame of the visualization can closely match the original Hubble image.

Most of these visualizations follow a similar pipeline. We begin by preparing the base files and breaking up the image, then follow it with 3D modeling, creating camera moves, generating stars, rendering, compositing, finding/creating music, rendering/compositing some more, and eventually releasing the final video to the public.

This is the idealized flow, but the process is often not linear. There is a lot of back and forth between the steps or working on multiple steps at once. This is all right! It’s part of the creative process when creating something with a lot of moving pieces.

Isolating the Visualization Components

Before we can bring any piece of AG Car into 3D space, we need to do a bit of preparation of base images that will be used throughout the viz process. This means that both the background stars and main star need to be removed, which leaves us with just the nebula portion of AG Car to work with.

Don’t worry, the background stars are brought back at a different step in the process (more on that later). The main star is isolated into its own image and the nebula is further divided into gas (red) and dust (blue) images. Dividing the nebula portion into these layers gives us better control when we move to the 3D modeling portion of the process.

Sometimes Space is a Problem

As you may or may not have noticed, space shows up as black. Nebulae have transparency. We want that transparency while creating the visualization, so you may be seeing the issue. To work around this issue we remove as much black as possible from the isolated images without compromising the visibility of the structures. Some of this is also handled in the texturing and compositing portion of the process.

Dividing up the Dust and Gas

After we have our base images isolated and space removed, we start the process of deciding what parts go where in space based on what we know about the shape and science of the object. Sometimes this part of the process can feel like trying to complete a very abstract, nebulous puzzle. It involves a lot of squinting at the screen, asking yourself “Is that where I think that is?”, drawing a line, and then repeating the process until you get it right.

The dust and gas images ended up being divided into roughly 10 layers each. Each of those layers are then saved out as individual images with varying levels of opacity to be used as textures later in the 3D modeling process.

So, we have these two images, one red, one blue. How do we know that we can bring them back together and have it look like the original release image?

The Magic of Screen Blending Mode

Basically, when stacking two images using screen blending mode in a photo processing or compositing program, the resulting color is always a lighter color. Screening with black leaves the color unchanged. Screening with white produces white. This comes in handy when you’re trying to overlay color (gas) on color (dust) or color (nebula) on black (space).

The 3D Modeling Process

The process of bringing AG Car into 3D space begins with creating a very basic toy model. In this case it’s two spheres that overlap and are slightly angled. To check the amount of overlap and angle, we overlaid it on the final AG Car image with the approaching and receding shells indicated.

Once we have the basic toy model we start the process of building the dust and gas models. For most visualizations we create one large model, but since we divided the gas and dust into two images, it allowed us to create two models and merge them in the compositing portion of our pipeline instead.

We first create planes in our 3D software that use scripting which allows them to change scale but still keep the textures aligned. It ends up looking like a pyramid of flat planes that face the camera in the scene. An important thing to note is how we place the main central star in the 3D scene. Earlier in the process we locate the the pixel in the press release image that is the exact center of the main central star using the diffraction spikes as a guide. This pixel lets us know where to align the first plane so that the exact center of the star sits at (0, 0, 0) in x, y, z space. All other planes in the scene are aligned to this star plane.

Each plane is then assigned a texture as an image that was created from the portion of the pipeline where we divided up the gas and dust. These textures are assigned to the planes from closest to the camera (the viewer) to furthest.

The textured planes are placed at different distances from the static camera based on the toy model. For example, we know that we would only see the dust in the closest lobe of AG Car to the viewer based on the science of the object, so the dust planes are only placed within the sphere closest to the camera.

Since we have a general idea of where the planes and textures are, we can now create the camera that will be used to fly though the 3D scene. For the AG Car visualization, we decided to showcase the back cone of the structure and try to push the feeling of parallax in the dust structures. This camera is later used to render (the process of turning information from a 3D scene into a 2D image) out the models for compositing.

Once the planes are moved to where we generally think they should be in space, we sculpt them. This is where we divide the planes into smaller sections where there is a lot of detail shown in the texture. We then push and pull areas of the plane towards or away from the camera to push the idea of the structures of AG Car. We make sure to only push or pull toward or away from the camera because if you shift portions of the planes to the sides, the textures will not align anymore and the first frame of the animation will not look like the press release image.

There is a lot of iteration in this phase of the pipeline. Our decided camera path shifted to the side and twisted, which meant that the textures were not viewed straight-on at all times. When the models are not viewed straight-on, you see “cracks” between the layers of the textures (shown later in the drafts images). This meant that each texture of each plane had to be brought into our photo processing software to have the gaps “painted” so that when the camera moves, it all appears as one object rather than a stack.

After all of the “gaps” are “painted,” the planes are placed where they should be, and the camera move is locked down, we render each model and the star plane separately along the same camera path. The end result is three sets of image frames, one for each structure.

A Side Note: Sleight-of-Hand from an Artist’s Perspective

When looking at the final visualization, most viewers wouldn’t be able to tell you the process used to make it. Most viewers wouldn’t be able to point out that the nebula and star are 21 stacked planes with varying transparencies rendered out to 3 sets of frames and then layered in compositing. Most viewers would tell you that it is a single object. This is where artistic sleight-of-hand comes into play. We work to find the best way to get an end product that balances scientific accuracy with aesthetics within the time frame and parameters that we have and use skillful tricks of the eye to achieve it when needed.

Synthetic Stars

The stars are generally not the focus of our visualizations, but they play a crucial role. As the camera flies across space to get a closer view, the stars carry all of the 3D effect as well as set the viewer up for the new perspective of the featured object. Having hundreds to thousands of stars distributed throughout space is fundamental to producing a 3D look and feel.

It is also important to create the look of the stars in the original Hubble image. The most basic way to do so is to cut the stars out of the image into small, individual images. We have used that technique in the past, but it can be time-consuming and relies on astronomical research software that we find cumbersome. Over the years, we have developed an alternative.

The Gaia catalog contains data on billions of stars across the sky. The data includes information relevant to visualization such as position, brightness in several colors, and a distance estimate (via the parallax value). Accordingly, we searched the Gaia catalog and found all the stars within a specified region on the sky around AG Car. We extracted the necessary data and did test visualizations to rotate, shift, and align the Gaia stars to the Hubble field of view.

Re-creating the look of Hubble stars requires Tiny Tim, an astronomical research software package that may be highly specific, but that we do not find cumbersome to use. When provided with information on the colors of a star, as well as the details of the instrument used for the observation, Tiny Tim produces a star image that is nearly identical to a Hubble image. We processed thousands of stars through Tiny Tim to fill out the visualization field.

While Gaia does include a distance estimate, we only partially use that information. Linear distance is not the objective in our visualizations, as a realistic flight across thousands of light-years of space would simply take way too long. We have developed a statistical distribution of stars that provides a suitable experience, and gets us to the featured object in a short traverse. It is somewhat of a logarithmic distance distribution, but heavily adjusted for artistic feel. The Gaia distance information was used to determine which stars are in front of AG Car, and which are behind.

Naturally, these “synthetic stars” are not perfect replacements for the stars deleted from the image. The Gaia catalog misses a good number of very faint stars that Hubble observes. Since the stars are not the focus, one has to look very closely to notice these missing stars. Also, the Gaia colors never correspond exactly to the colors Hubble observed. The red, green, and blue channels each need to be adjusted independently in order to recreate the look of the press release image.

In creating the restored stars for the 3D model, four streams of star processing are extracted from the Gaia data, turned into images via Tiny Tim, given distances from the statistical model, and output as floating billboards for the 3D model. The foreground and background stars are rendered separately, and later composited with the nebula.

It All Comes Together

After we have the five separate renders from our 3D software of the main star, gas, dust, background stars, and foreground stars, we move on to the process where we bring them all together in our compositing software.

Each render is stacked in accordance of where they are in space and screen mode blending is used for relevant layers to get the first frame of the animation to look like the press release image. This is where we also make tweaks to transparency and color as needed to get the animation to look closer to the release image.

We often go through multiple draft composites throughout the visualization process to use as references for which areas need tweaking. Things don’t have to look good right away! Drafts are meant to give you a better idea of what needs to change and over time they get more refined (I mean, look at Draft 2 versus Draft 5).

After our models, camera path, textures, stars, music, and final composite are given the “thumbs up,” the final piece is ready to be rendered out to video, combined with mood-setting music, and released to the public.

Journey to the Fly Through

While the visualization at this point is complete and ready for take-off, we often like to provide context for the object we are about to explore, rather than simply dropping the viewer off in an unknown region in space. To give this context, we usually preface the visualization or Hubble image with a “zoom-in” video which shows you where an object is on the sky using a wide-field image, and then proceeds to dive down further into the image, ending with the narrow field of the Hubble observation, or in this case, the visualization.

In order to create a zoom-in, you need at least three different images at varying fields of view you can transition between to preserve the resolution as you travel inward (alas, there is no endless “enhance” button). In this case, we used a ground-based photograph provided by Akira Fujii for our starting image, a region of the Digitized Sky Survey for our intermediate image, and lastly the first frame of our AG Car visualization. These images must be aligned and adjusted to match each other as closely as possible in contrast and color, so the transitions between the images feel smooth. The images are then nested in such a way that they scale upwards simultaneously, creating the zoom-like motion. In order to make this motion feel constant, the images are scaled exponentially, otherwise the zoom would feel very fast in the beginning and very slow at the end.

Once the speed of the zoom is adjusted to feel just right, we then can add overlays of constellation drawings and text on the wide-field sky image. Finally, we merge the visualization and zoom-in into one video with the delicacy and precision required to dock a Dragon capsule to the International Space Station and create music for the combined video.

Musical Notes
by Joe DePasquale

One of the perks of working in scientific visualization and being a musician on the side, is the occasion where both skills can come together in the service of a visualization project. There is a wonderful synergy that happens when you’re intimately familiar with the nuts and bolts of a project, and then have the opportunity to take a step back from all of that, see the big picture, and help bring all of its elements together with music that hopefully builds on and reinforces the inspiration found within the imagery and visualization.

As the animation of AG Car started to come together, I was inspired to write a short piece of music that tapped into the majesty of this nebula and its brilliant, luminous star. The animation starts with a ground-based, wide-field view of the sky in the area of the constellation Carina and then zooms directly to the Hubble image. I wanted the music in those initial frames to have the feeling of lifting you off of the planet as if on your own personal rocket ship. Heavy strings establish a theme which succumbs to the beautiful desolation of space as we make our way across the galaxy. As we approach AG Car, the strings and airy guitar melt together in the reverberation of the universe and give way to peaceful, muted piano elaborating on the established theme.

It is at this point, when we transition from two-dimensional imagery to a three-dimensional model of the stars and nebula around AG Carinae, the music takes a turn with the introduction of an arpeggiated synthesizer outlining the chords of our peaceful piano, adding a new dimension to the music. As the last of the nearby stars fly past our rocket ship, a new twist on this arpeggiated theme is introduced, echoing the stellar winds of AG Car in my mind’s eye (and ear!). Finally, the strings return, reminding us that we are, in fact, firmly planted on terra firma, but thanks to Hubble and a skilled team of science visualizers, we can take these fantastic journeys through space. As Carl Sagan famously said, “we are made of star stuff” and it was in that spirit that I titled this piece “Luminous Beings.”

We hope you enjoy a flight to AG Car!