“We have to be forward-looking”

The JPEG standard, introduced in 1994, is now the predominant image-sharing format for the web and social media, with billions of images transferred under the format each day. The standard’s compression algorithms have become so widespread that nobody stops to wonder who administers them. But a committee of international experts is busy behind the scenes, updating the algorithms regularly in response to technological and societal developments. For instance, several versions of the JPEG standard have been introduced in recent years. Prof. Ebrahimi, a professor of multimedia signal processing at EPFL (School of Engineering), was appointed chair of the JPEG standardization committee in 2014. He’ll be awarded the 2022 SMPTE Progress Medal for his decisive work in developing new standards, such as in the areas of artificial intelligence and DNA storage. The award will be handed out at a ceremony in Los Angeles on 27 October 2022. The prestigious SMPTE Progress Medal has been given out every year since 1935.

Were you expecting to receive this award? How did you learn that you’d won?

I didn’t expect it at all. Unlike with other awards, SMPTE Medal winners aren’t nominated by their peers. The selection is done through a committee, whose lips are sealed. In fact, I almost missed the email telling me I’d won! I was checking my messages while walking home one day, and I thought it was spam. Fortunately, something in the email caught my eye – it was written by someone who seemed to know a lot about my work.

What does your job as the chair of the JPEG standardization committee consist of?

My work has been focused on three main topics throughout my career: coding visual information, especially in JPEG and MPEG formats; data security, primarily in the areas of rights protection, watermarks, authentication, data integrity and steganography [methods for concealing messages]; and compressed image quality, or how much data are lost during compression. All compression algorithms involve some degree of loss. And before artificial intelligence came into the picture, the idea was to “sacrifice” the data that weren’t visible to the human eye. But a set of standards had to be established so that the image transfer would be seamless across the entire chain, from cameras and applications to end users. That’s where the standards set by JPEG – which stands for Joint Photographic Experts Group – come in. For example, machines don’t automatically know what constitutes a high-quality image for humans; those characteristics have to be programmed in by algorithms. And when it comes to data security, standards are needed so that people know when an image has been modified. How can we make sure people are aware that the data they’re viewing in JPEG format have been processed or manipulated by an AI program? Our job is to look ahead to new technology on the horizon and update our format accordingly.

The JPEG AI standard is scheduled to come out next year. What will it improve?

Artificial intelligence has really taken hold over the past ten years and can now be found just about everywhere – in our cars, smartphones and security systems, for example. At JPEG, everything we’ve done for the past 30-plus years has been intended for humans. But now, that level of data is seen as inadequate along the entire chain, starting with cameras. AI has brought about a complete paradigm shift. The goal going forward is to develop systems that can leverage the full capacity of machines, too. We can’t draw a line anymore between humans and machines, since in many applications, the same system is used by both. For example, humans use smartphone cameras to take selfies, while machines use them for facial recognition. So we had to find a way for algorithms to meet both needs – to deliver enhanced compression, so that images appear sharp to the human eye but without impacting machine vision performance. As you mentioned, the JPEG AI standard will be released by end-2023. We always have to be one step ahead with our standards, but since they take three to four years to finalize, we’ve got to be on the lookout for the next breakthrough. That’s also what makes my job so interesting: we have to be forward-looking. When we develop a JPEG standard, we create an ecosystem that opens up opportunities for users – even before the associated technology is widely adopted. The first JPEG standard was developed well before the corresponding applications existed. In fact, it’s thanks to the JPEG format that the applications could be created.

After JPEG AI is released next year, what will be your next big challenge?

The fundamental challenge we face is always the same. An ever-growing amount of data is being collected and stored, but the size of batteries and storage systems is subject to space and cost constraints. So the data have to be compressed, but when it comes to images, there’s a limit to the compression – otherwise, the image will look like it’s of poor quality. We’re currently working on a JPEG DNA standard for new DNA-based storage technology. It’s basically an efficient image coding format for creating artificial DNA molecules.

JPEG standards, including the different versions, have a near monopoly in areas like the internet and social media, but they were developed by a public-sector, non-profit organization. How do you explain that?

One big advantage is that JPEG was the first internationally standardized format for image compression, and it was released in 1994 just as the internet was gaining traction. Internet developers therefore naturally adopted it. The fact that our format is free to use also helps. And last but certainly not least, I would point to the excellent quality of JPEG standards. I’ve spent my entire career trying to continually improve compression – and there’s still a lot more to be done – but I can safely say JPEG algorithms are ahead of their time. Even after 30 years, experts are struggling to come up with something better. Some joke that the JPEG format was engineered by “aliens from the future!”

What makes it likely that a standard will be widely adopted? Do you think that’ll be the case with JPEG Pleno, your new standard for virtual and immersive reality?

While very few people walk around today with augmented reality glasses, our standard is ready for when immersive technology becomes widespread. It’s like when developers created Facebook and Instagram using the regular JPEG standard, before those applications were commonly known. Two big factors in getting a new standard adopted are, first, not having a profit-seeking motivation behind it, and second, designing it to work on all types of devices and operating systems. Then the big multinationals can decide whether to use it. But there’s also pressure from consumers who don’t want to be locked into a given system – they want to be able to switch brands seamlessly, with no interruption in how they use their favorite apps. Consumer watchdog groups, like those established by the EU, have already taken steps against the tech giants that create too many barriers, such as Apple and Microsoft.

What attracted you to signal processing, a field that at the time was still in its infancy?

It was a career I chanced upon. It wasn’t a childhood dream of mine – I could’ve done something else. When I was a student at EPFL, I hesitated between specializing in robotics, another field I had strong skills in, and signal processing. But a PhD position opened up in signal processing so I took it – and haven’t looked back. When you’re young, it’s hard to be sure of what you want do until you’ve tried it out and invested your time and energy.