FLOSS Project Planets
PyBites: Introducing eXact-RAG: the ultimate local Multimodal Rag
Exact-RAG is a powerful multimodal model designed for Retrieval-Augmented Generation (RAG). It seamlessly integrates text, visual and audio information, allowing for enhanced content understanding and generation.
In the rapidly evolving landscape of the Large language model (LLM), the quest for more efficient and versatile models continues unabated.
One of the latest advancements in this realm is the emergence of eXact-RAG, a multimodal RAG (Retrieval Augmented Generation) system that leverages state-of-the-art technologies to deliver powerful results.
eXact-RAG stands out for its integration of LangChain and Ollama for backend and model serving, FastAPI for REST API service, and its adaptability through the utilization of ChromaDB or Elasticsearch.
Coupled with an intuitive user interface built on Streamlit, eXact-RAG represents a significant leap forward in LLMs capabilities.
A Step back: what is a RAG?Retrieval Augmented Generation, or RAG, is an architectural approach that can improve the efficacy of Large Language Model (LLM) applications by leveraging custom data. This is done by retrieving data/documents relevant to a question or task and providing them as context for the LLM. RAG has shown success in support chatbots and Q&A systems that need to maintain up-to-date information or access domain-specific knowledge.
RAG process schema. Source: neo4j.comAs the name suggests, RAG has two phases: retrieval and content generation. In the retrieval phase, algorithms search for and retrieve snippets of information relevant to the user’s prompt or question. In an open-domain, consumer setting, those facts can come from indexed documents on the internet; in a closed-domain, enterprise setting, a narrower set of sources are typically used for added security and reliability.
Understanding eXact-RAGAt its core, eXact-RAG combines the principles of RAG, which focuses on retrieval-based conversational agents, with multimodal capabilities, enabling it to process and generate responses from various modalities such as text, images, and audio.
This versatility makes eXact-RAG well-suited for a wide range of applications, from chatbots to content recommendation systems and beyond.
Technologies Powering eXact-RAG- LangChain and Ollama: LangChain and Ollama serve as the backbone of eXact-RAG, providing robust infrastructure for model development, training, and serving.
LangChain offers a comprehensive suite of tools for natural language understanding and processing, while Ollama specializes in multimodal learning, enabling eXact-RAG to seamlessly integrate and process diverse data types. - FastAPI for REST API Service: FastAPI, known for its high performance and simplicity, serves as the interface for eXact-RAG, facilitating seamless communication between the backend system and external applications.
Its asynchronous capabilities ensure rapid response times, crucial for real-time interactions. - ChromaDB or Elasticsearch: eXact-RAG offers flexibility in data storage and retrieval by supporting both ChromaDB and Elasticsearch.
ChromaDB provides a lightweight solution suitable for simpler tasks, while Elasticsearch caters to more complex operations involving vast amounts of data. This versatility enables users to tailor eXact-RAG to their specific needs, balancing performance and scalability accordingly.
The user interface of eXact-RAG is built on Streamlit, a popular framework for creating interactive web applications with Python. Streamlit’s intuitive design and seamless integration with Python libraries allow users to interact with eXact-RAG effortlessly.
Through the interface, users can input queries, explore results, and interact with generated content across various modalities, enhancing the overall user experience.
Related article: From concepts to MVPs: Validate Your Idea in few Lines of Code with Streamlit
Applications of eXact-RAGThe versatility of eXact-RAG opens up a myriad of applications across different domains:
- Conversational Agents: eXact-RAG can power chatbots and virtual assistants capable of engaging users in natural and meaningful conversations, leveraging both text and multimedia inputs.
- Content Recommendation: By analyzing user preferences and behavior, eXact-RAG can recommend personalized content, including articles, videos, and images, tailored to individual tastes and interests.
- Information Retrieval: eXact-RAG excels at retrieving relevant information from large datasets, making it invaluable for tasks such as question answering, document summarizing, and knowledge base retrieval.
The first step to use eXact-RAG is to run your preferred LLM model using Ollama or get an OpenAI token. Both are supported by the RAG and to be sure to configure them rightly, it’s necessary to fill the settings.toml with the preferred options. Here an example:
[embedding] type = "openai" api_key = "" chat.model_name = "gpt-3.5-turbo" ... [database] type = "chroma" persist_directory = "persist" ...In this example it’s possible to configure the LLM model and the vector database in which store the embeddings of your local data.
eXact-RAG is a multimodal RAG, for this reason it can ingest different kind of data like audio files or images. It is possible to choose, in the installation phase, which “backends” to install:
poetry install # -E audio -E image- audio extra will install openai-whisper for speech-to-text
- image extra will install transformers and pillow for image captioning*
* this feature gives the possibility to process images even if the user has not the (hardware) possibility to run a Vision model like llava locally but still wants to pass images as data.
Now the job is done! The following command
poetry run python exact_rag/main.pystarts the server and at http://localhost:8080/docs it is showed the OpenAPI document (swagger) with all the available endpoints.
DemoeXact-RAG was built as a server for multimodal RAGs but we provide also a user interface just for demo purposes to test all the features.
To run the UI just use the command:
Now, the page at http://localhost:8501 will show a chat interface like in the following example:
ConclusioneXact-RAG represents a significant advancement in the field of multimodal RAG systems, offering unparalleled versatility and performance through its integration of cutting-edge technologies.
With its robust backend powered by LangChain and Ollama, flexible data storage options, and user-friendly interface built on Streamlit, eXact-RAG is poised to revolutionize various applications of natural language processing and multimodal learning.
As the demand for sophisticated LLM solutions continues to grow, eXact-RAG stands ready to meet the challenges of tomorrow’s digital landscape.
C.J. Collier: Signed NVIDIA drivers on Google Cloud Dataproc 2.2
Hello folks,
I’ve been working this year on better integrating NVIDIA hardware with the Google Cloud Dataproc product (Hadoop on Google Cloud) running the default cluster node image. We have an open bug[1] in the initialization-actions repo regarding creation failures upon enabling secure boot. This is because with secure boot, kernel driver code has its signature verified before insmod places the symbols into kernel memory. The verification process involves reading trust root certificates from EFI variables, and validating that the signatures on the kernel driver either a) were made directly by one of the certificates in the boot sector or b) were made by certificates which chain up to one of them.
This means that Dataproc disk images must have a certificate installed into them. My work on the internals will likely start producing images which have certificates from Google in them. In the meantime, however, our users are left without a mechanism to have both secure boot enabled and install out-of-tree kernel modules such as the NVIDIA GPU drivers. To that end, I’ve got PR #83[2] open with the GoogleCloudDataproc/custom-images github repository. This PR introduces a new argument to the custom image creation script, `–trusted-cert`, the argument of which is the path to a DER-encoded certificate to be included in the certificate database in the EFI variables of the disk’s boot sector.
I’ve written up the instructions on creating a custom image with a trusted certificate here:
https://github.com/cjac/custom-images/blob/secure-boot-custom-image/examples/secure-boot/README.md
Here is a set of commands that can be used to create a general purpose GCE disk image with the certificate inserted into EFI.
wget 'https://github.com/cjac/custom-images/raw/secure-boot-custom-image/examples/secure-boot/create-key-pair.sh' bash create-key-pair.sh cacert_der=tls/db.der # The Microsoft Corporation UEFI CA 2011 ms_uefi_ca="tls/MicCorUEFCA2011_2011-06-27.crt" test -f "${ms_uefi_ca}" || \ curl -L -o ${ms_uefi_ca} 'https://go.microsoft.com/fwlink/p/?linkid=321194' JSON="$(gcloud compute images --project debian-cloud list --format json)" SRC_IMAGE_NAME="$(echo ${JSON} | jq -r '.[] | .name' | grep -i ^debian-12-bookworm-v)" NEW_IMAGE_NAME="debian12-with-db-key-list-${USER}-$(date +%F)" SRC_DISK_ZONE="${ZONE}" gcloud -q compute images create "${NEW_IMAGE_NAME}" \ --source-disk "${SRC_IMAGE_NAME}" \ --source-disk-zone "${SRC_DISK_ZONE}" \ --signature-database-file="${cacert_der},${ms_uefi_ca}" \I’d love to hear your feedback!
[1] https://github.com/GoogleCloudDataproc/initialization-actions/issues/1058
[2] https://github.com/GoogleCloudDataproc/custom-images/pull/83
Daniel Lange: Fixing esptool read_flash above 2MB on some cheap ESP32 boards
esptool, the Espressif SoC serial bootloader utility, tends to dislike cheap Flash chips attached to the various incarnations of the ESP32 chip family. And it seems to dislike them even more when running esptool on Linux than on other OSs.
The common error mode is seeing it break at the 2MB barrier when trying to dump (esptool read_flash) a 4MB flash configuration.
esptool -p /dev/ttyUSB0 -b 921600 read_flash 0 0x400000 flash_dump.binwill fail with
esptool.py v4.7.0 Serial port /dev/ttyUSB0 Connecting.... Detecting chip type... ESP32 Chip is ESP32-D0WD-V3 (revision v3.1) Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None Crystal is 40MHz [..] Detected flash size: 4MB [..] 2097152 (50 %) A fatal error occurred: Failed to read flash block (result was 01090000: CRC or checksum was invalid)typically at the 2MB barrier.
I found the solution in a rather unrelated esptool Github issue:
Create an esptool.cfg file in the project directory (from where you will run esptool):
[esptool]timeout = 30
max_timeout = 240
erase_write_timeout_per_mb = 40
mem_end_rom_timeout = 0.2
serial_write_timeout = 10
The timeout = 30 is the setting that fixed reading flash memory via esptool read_flash for me.
When your esptool.cfg is read, esptool will tell you so in its second line of output:
$ esptool flash_id esptool.py v4.7.0 Loaded custom configuration from /home/dl/[..]/Embedded_dev/ESP-32_Wemos/esptool.cfg Found 1 serial ports Serial port /dev/ttyUSB0 Connecting...... [..]Thank you Radim Karnis and wibbit from the Github issue linked above.
Kdenlive 24.05.1 released
The first maintenance release of the 24.05 series is out fixing issues in the spacer tool, effects and compositions, subtitle management and project settings to name a few. We addressed recently introduced crashes and freezes, including fixing the undo/redo track insertion and multiple track insertion issues. This version also improves AppImage packaging and enables notarization for macOS.
Full changelog
- Don’t try renaming sequence on double click in empty area of timeline tab bar. Commit.
- Fix deletion of wrong effect wihh multiple instances of an effect and group effects enabled. Commit.
- Fix single selected clip disappearing from timeline when dragging a new clip in timeline. Commit.
- [cmd rendering] Ensure proper kdenlive_render path for AppImage. Commit.
- Fix freeze/crash on undo/redo track insertion. Commit.
- Fix crash on undo/redo multiple track insertion. Commit.
- Project settings: don’t list embedded title clips as empty files in the project files tab. Commit.
- Fix undo move effect up/down. On effect move, also move the active index, increase margins between effects. Commit.
- Fix removing a composition from favorites. Commit.
- Properly activate effect when added to a timeline clip. Commit.
- Fix spacer tool can move backwards and overlap existing clips. Commit.
- Fix crash deleting subtitle when the file url was selected. Commit. Fixes bug #487872.
- Fix build when using openGLES. Commit. Fixes bug #483425.
- Fix possible crash on project opening. Commit.
- Fix extra dash added to custom clip job output. Commit. See bug #487115.
- Fix usage of QUrl for LUT lists. Commit. See bug #487375.
- Fix default keyframe type referencing the old deprecated smooth type. Commit.
- Be more clever splitting custom ffmpeg commands around quotes. Commit. See bug #487115.
- Fix effect name focus in save effect. Commit. See bug #486310.
- Fix tests. Commit.
- Fix selection when cutting an unselected clip under mouse. Commit.
- Fix loading timeline clip with disabled stack should be disabled. Commit.
- Fix crash trying to save effect with slash in name. Commit. Fixes bug #487224.
- Remove quotes in custom clip jobe, fix progress display. Commit. See bug #487115.
- Fix setting sequence thumbnail from clip monitor. Commit.
- Fix locked track items don’t have red background on project open. Commit.
- Fix spacer tool doing fake moves with clips in locked tracks. Commit.
- Hide timeline clip status tooltip when mouse leaves. Commit
The post Kdenlive 24.05.1 released appeared first on Kdenlive.
PyBites: What is the Repository Pattern and How to Use it in Python?
The repository pattern is a design pattern that helps you separate business logic from data access code.
It does so by providing a unified interface for interacting with different data sources, bringing the following advantages to your system:
- Flexibility: You can swap out your data source without changing business logic.
- Maintainability: It’s easier to manage and test different storage backends.
- Decoupling: You reduce tight coupling between your application and data access layer.
Let’s use Python and sqlmodel (PyPI) to demonstrate this pattern (code here):
from abc import ABC, abstractmethod from sqlmodel import SQLModel, create_engine, Session, Field, select # Define the model class Item(SQLModel, table=True): id: int = Field(default=None, primary_key=True) name: str # Repository Interface class IRepository(ABC): @abstractmethod def add(self, item: Item): pass @abstractmethod def get(self, name: str) -> Item | None: pass # SQLModel implementation class SQLModelRepository(IRepository): def __init__(self, db_string="sqlite:///todo.db"): self.engine = create_engine(db_string) SQLModel.metadata.create_all(self.engine) self.session = Session(self.engine) def add(self, item: Item): self.session.add(item) self.session.commit() def get(self, name: str) -> Item | None: statement = select(Item).where(Item.name == name) return self.session.exec(statement).first() # CSV implementation class CsvRepository(IRepository): def __init__(self, file_path="todo.csv"): self._file_path = file_path def add(self, item: Item): with open(self._file_path, "a") as f: f.write(f"{item.id},{item.name}\n") def get(self, name: str) -> Item | None: with open(self._file_path, "r") as f: return next( ( Item(id=int(id_str), name=item_name) for line in f if (id_str := line.strip().split(",", 1)[0]) and (item_name := line.strip().split(",", 1)[1]) == name ), None, ) if __name__ == "__main__": repo = SQLModelRepository() repo.add(Item(name="Buy Milk")) sql_item = repo.get("Buy Milk") # Swap out the repository implementation csv_repo = CsvRepository() csv_repo.add(Item(id=1, name="Buy Milk")) csv_item = csv_repo.get("Buy Milk") print(f"{sql_item=}, {csv_item=}, {sql_item == csv_item=}") # outputs: # sql_item=Item(name='Buy Milk', id=1), csv_item=Item(id=1, name='Buy Milk'), sql_item == csv_item=True- First we define the Item model using SQLModel. You can also use SQLAlchemy or any other ORM, but I find SQLModel a bit easier to use and I like its integration with Pydantic.
- Next we define the IRepository interface with the add and get methods, making them required for any class that inherits from it. This is done using Python’s Abstract Base Classes (ABCs) and applying the @abstractmethod decorator. This is a way to “enforce a contract” and a crucial part of the repository pattern (see also the tests). If you don’t implement these methods in a subclass, Python will raise an error. This is a way to ensure that all repository classes have the same interface, even if they use different storage. To learn more about ABCs, check out our article).
- Then we implement the SQLModelRepository and CsvRepository subclasses, which inherit from IRepository. These classes implement the add and get methods, which are required by the IRepository interface. This is where we define the data access logic for each of the storage. The SQLModelRepository uses SQLModel to interact with a SQLite database, while the CsvRepositoryinteracts with a CSV file. Same interface, different storage backends.
- Finally, we demonstrate how to use the repository pattern by adding an item to both the SQL and CSV repositories and then retrieving it.
Note: In this implementation, we did not add error handling to keep the example relatively small. You should ensure that if something goes wrong (e.g., database connection issues or file access errors), the program can handle the error gracefully and provide useful feedback.
The example might be a bit contrived, but it shows how we can leverage the repository pattern in Python.
Again, the advantage is that we have a flexible design that allows us to swap out the data access implementation without changing the business logic. This makes our code easier to test and maintain.
Have you used this pattern yourself and if so, how? Hit us up on social media and let us know: @pybites …
DrupalEasy: Visual Debugger module: a modern take on an old idea
If you've been a Drupal developer since before the time of modern Drupal (pre-Drupal 8,) then you probably remember the Theme Developer module. This module is/was used to figure out theme template usage and possible overrides in Drupal 7.
With modern Drupal, this same information is available directly in the HTML source code of a file when Twig development mode is enabled.
The Visual Debugger module by Marcos Hollanda combines the best of the present with the best of the past to surface template information directly in the user interface without having to dig into the page's HTML.
Pre-impressionsI will admit that before I installed the module, I was doubtful that I'd be interested in using it. Marcos had pinged me on Slack asking me to take a look at it, and I figured it would be a nice exercise for DrupalEasy Office Hours, which it was.
Much to my surprise though, I really like using it.
Basic usageThere's not much to say about how to use this module - it works just like you'd expect. Use Composer to get the code, enable the module, then enable Twig development mode and you're done. By default it will appear on each non-admin page of your site, with a handy activate/deactivate icon. There's really not too much to say about its usage 😀.
Current statusThe module is in alpha status with several planned features not yet implemented. What is implemented so far appears to be solid and useful. The basic functionality that I expected is there. Click on any section of the page, and the module will display the template file that is used to render that section as well as a list of possible template file override names. This is the same information that Twig developer mode displays in HTML comments.
Marcos let me know that other planned features include theme hook suggestions and displaying caching information; (similar to the Cache review module, perhaps?)
There are other challenges to overcome as well, including how to handle selecting overlapping elements. I did make one very minor suggestion to relocate the module's configuration page which Marcos quickly implemented 🤘🏼.
Who is this module for?I would definitely put this module in the category of a developer tool for folks who are either new to Drupal theme development or people who prefer a UI instead of scrolling through raw HTML.
I plan on following the progress of this module closely, in hopes that it will be suitable to use during the next semester of Drupal Career Online, where we have a theming lesson that could definitely benefit from this module.
1xINTERNET blog: 10 React use cases at 1xINTERNET
React is one of the primary technologies in our technology stack. This article explores our React use cases, outlining their purpose, technical aspects and implementation examples.
The Drop Times: Driving Drupal Forward: Suzanne Dergacheva on the Strategic Rebranding of Drupal
Tag1 Consulting: Migrating Your Data from Drupal 7 to Drupal 10: Generating migrations with Migrate Upgrade
Series Overview & ToC | Previous Article | Next Article - coming June 27th In the previous article we performed an automated migration via the administration interface with tools provided by Drupal core out of the box. This gave us the opportunity to familiarize ourselves with the different modules that assist with the task, settings that affect the migration, and the results when there are no customizations in place. Today we will leverage the Migrate Plus and Migrate Upgrade modules to generate migrations from the command line, which we can then pick from and customize. Automated migration using the Migrate Upgrade module Let’s start by listing the modules that we will use to run the automated migration from the command line: * Migrate: consists of the core API that can be used to import data from any source. * Migrate Drupal: allows connecting to a Drupal 6 or 7 site to import configuration and content. * Migrate Plus: provides extra source, process, and destination plugins; handles migrations as configuration entities; allows reusing of configuration via migration groups; and more. It is a dependency of Migrate Upgrade and a very useful module on its own. * Migrate Upgrade: provides a Drush...
Read more mauricio Thu, 06/20/2024 - 05:15Talk Python to Me: #467: Data Science Panel at PyCon 2024
Matt Layman: Password Resets and Signal Handling - Building SaaS #194
Mario Hernandez: Managing image embeds with Drupal media
Allowing your content creators to embed images in text fields is a big risk if you don't have the right measures in place to get properly rendered images without jeopardizing your site's performance. We faced this issue first-hand with embedded images due to not using the right configuration and this lead to extremely large images being rendered. In this post I'll go over the techniques I took for addressing those issues and set up a system for image embeds that is solid and performant.
I started by writing a seven-part guide on how to setup responsive images. In this post I'll focus on image embeds. If you followed or read the responsive images guide, you should be able to take advantage of some of the work we did there in this post. The guidelines covered here include:
- Defining requirements
- Image styles
- Media view modes
- Text format configuration
Before you write the first line of code or set the first drupal configuration for this issue, you need to have a clear understanding of your requirements. Here is a summary of my requirements:
-
Only certain user roles can embed images
This means we need to figure out if the text formats used in our site will allow us to set the restrictions we need. Otherwise we may need to create or edit a text format for our target user roles.
-
Users need to be able to choose the image size and aspect ratio when embedding images
We defined the image sizes and aspect ratios and assigned names that were user-friendly for non-technical users. We came up with name options we think our users will find easy to work with such as:
- Small square, Small portrait, Small rectangular
- Medium square, Medium portrait, Medium rectangular, Medium rectangular wide
- Large square, Large rectangular, Large rectangular wide
- Extra large square, Extra large rectangular, Extra large rectangular wide
-
If no option is selected by users, set a default image size
For the default option when no selection is made by the user, we decided to use the Medium rectangular option. This has an aspect ratio of 3:2 and it measures about 720x480.
-
Existing Media items need to be available for embedding
This was a tricky one because my original inclination was to create a new Media type so we can isolate all configuration for its view modes and not overpopulate our default Media type. However, this ended up not working for us because when you limit your image embeds to only use a new Media type, you don't get access to any of the media items (images), that have already been uploaded to the Media library using other media types. Ultimately we ended up using Drupal core's Media type, Image, and our dev team had to compromise on having a very busy list of view modes for this media type.
-
Images need the ability to be cropped wihin the Media page
Since most of our images already provide the ability to be cropped at different aspect ratios, using the core Media type in the previous bullet point made this an easy solution.
It all starts with image styles. I'm not going to go over how to create image styles, you can read my post Image styles in Drupal. The one thing I am going to repeat however is the importance of creating reusable image styles. Reusable image styles can help you reduce the number of image styles you create while providing the flexibility you need with each use case.
Image styles are key as each of the size options we defined above translate into image styles. So Small square for example, is an image style that is defined as 1:1 (250px). Medium rectangular would be something like 3:2 (720x480), etc. You may be wondering, how do you plan on using fiendly names for your content editors when your image styles names are not very friendly? Great question. Since we are using Drupal's Media, content editors do not interact directly with image styles, they do with Media view modes and this is where we will use more friendly names.
Media view modesView modes are one of Drupal's powerful features. Being able to display content is different ways with little effort can turn a simple website into a dynamic content hub. The example I always give when someone asks me what view modes are or how do they work is the Amazon website. When you are viewing a product in amazon.com, or most retail websites for that matter, you will notice that the same product or similar ones appear all over the page but in slightly different ways, with different fields or styles. See the page below for an example.
The image above shows many ways in which a product can be displayed. I've numbered each display.
In Drupal, every entity such as content types, media types, blocks, etc., offer the ability to create view modes. For the purpose of image embeds, we will create a Media type view mode for each image style we plan on using. The view modes is what content editors will interact with when choosing an image size or aspect ratio during the image embed process. This is where we will use the user-friendly names we defined earlier. Let's go over how this relationship between view modes and image styles works for image embeds.
Configure view modes for the Image media type-
In your Drupal site, create an image style for each image size option you wish to provide to users when embedding images.
-
Next, create a Media view mode for each image style (/admin/structure/display-modes/view). Very iimportant: Remember the view mode's label (name) is where we are going to use the friendly name (i.e. Medium rectangular (720x480)). I like to keep the machine name similar to the label so it's easier to debug or identify in code (i.e. medium_rectangular_720x480).
-
Now, let's tie 1 & 2 together:
- Go to the media type you plan on using for media embeds (/admin/structure/media/manage/image/display). I am using Drupal core's Image media type.
- Scroll down and expand the Custom display settings fieldset.
- Check each of the view modes you created in step 2 and click Save.
-
Now click each of the view modes and update the image field to use the respective/matching image style.
View modes and image styles are all configured. Now let's configure the Text format that authorized users will use to embed images.
- Go to the Text formats and editors page (/admin/config/content/formats)
- Click Configure next to the text format you plan on using (i.e. Full HTML)
- Ensure the right user roles are selected
- Within the Toolbar configuration section, drag the Drupal media button from the Available buttons options to the Active toolbar section. You could probably remove the original insert image button since you won't be using it.
- Scroll to the Enabled filters section and check the Embed media checkbox
- Scroll to the Filter settings section and set the following:
-
Default view mode: This is the default display that will be used if content editors don't pick an option when embedding images. Select any of the view modes that represents the image size you want to use as default.
-
Media types selectable in the Media Library: Select the Media type you plan on using. In my case is Image.
-
View modes selectable in the 'Edit media' dialog: Finally, select each of the view modes you created in the previous section. FYI: View modes will be sorted in alpha order by their machine name. In my case I had to prefix some of the machine names with either "a" or "b" so the list of options for the users to choose from would be nicely organized by their label name. See screnshot below.
-
Click Save configuration
-
Now that we've completed all the configuration we should be able to take it for test drive.
- Go to any page where there is a text field with a WYSIWYG editor
- Make sure you are using the right text format by selecting it at the bottom of the text field where you want to embed an image
- Click the Insert media button from the editor's toolbar
- Select or upload the image you'd like to embed
- When the image has been inserted, click on it and you should see several options of actions you can do with the image. Things like align the image, add a caption, link it, and you should also see a selection box listing all the view modes you created.
- After making your selection you should immediately see the image size/aspect ratio change to the one you selected. When you are happy with your selection, click the Save button to save your page.
Important: Depending on your site's configuration, the options for changing your image size may look different than mine. In my case, I am only using Drupal's core modules and this is how the options look for me:
In closingPutting a system like this for your image embeds will give you the piece of mind that content editors have options to choose how big or small they would like images to be displayed, and from a performance point of view, if your image styles are done properly, you can rest assurred that bloated images will never be rendered because you have put the guard rails in place to avoid this from happening.
I hope you found this article useful and can put these techniques to use in your own Drupal project. Happy New Year! 🎉 🎊 🎆 👋
Mario Hernandez: Responsive images, wrapping up
As far as image resolution switching, all the work we need to do is done. We will proceed to creating a quick Drupal view to display a list of news articles each of which will have a responsive image. When we finish with that, we will do the last pending thing, configuring responsive image styles using the <picture> element. You didn't forget about the picture element, did you?
Creating a new Drupal view for news articlesOut of the box Drupal already comes with a view that shows a list of content, but in the Olivero theme this view is not displayed the way we need it, so we will create our own view.
- From the admin toolbar, click, Structure | Views
- Click Add view
- Give the new view any name you wish (i.e. News, Latest News, etc.)
- In the View settings select to show Content of type Article
- In Page settings check Create a page
- Page title can be anything you wish
- Type news (lower case) as the path
- Under Page display settings chose Unformatted list of teasers
- Scroll and click Save and edit
- Scroll down and click Save
Two important things in the view above:
- The path is the url where the content list will be available (i.e. https://my-site.com/news).
- Using Teaser allows us to make sure of all the previous configuration we've done for responsive images.
With the view now in place, go to the new page we created, /news. This should show a list of news articles. Don't worry if the list does not look like the screenshot we showed earlier. The main thing is that you get a list of articles with images. The images in each article should reflect the configuration we setup for responsive images. You can check this by inspecting the images and you should see many of the attributes we configured. See example below.
- First, the image is rendering using the img tag and not <picture>
- The img tag uses the srcset and sizes attributes
- We can see the diferent image styles we created
That's it! We have completed all the steps for managing responsive images using the resolution switching approach. Now we will complete this series by going back to creating a new responsive image style but this time using the <picture> element.
Responsive image style using art directionI opted to leave this approach for last because it requires a couple of extra tools that are not part of Drupal core. As previously mentioned, art direction requires that each device gets an image cropped differently to ensure better user experience. Although Drupal provides a cropping "effect" within image styles, it is an automatic cropping process and cannot be customized for each image. This may not always work because we don't know what the focal point for each image we need cropped may need to be when trying to achieve art direction. For example, I may upload an image that I need to crop so the focal point is on the top-left corner, but next time I may upload an image where the focal point should be the bottom-right corner. Focal point refers to picking the area that is the most important to display specially on small devices.
What we need is to be able to manually crop each image we intend to do art direction with as we add them to the site. The good news is that within the Drupal echosystem we have many tools to choose from. In the interest of time, I'm going to simply discuss the tools you need to be able to manually crop your images on demand, but will not go into detail about their configuration or settings as those modules/tools have good documentation you can follow for proper configuration.
Here are the tools we will be working with:
- Crop API: On its own it does nothing, but in combination with other tools like Image widget crop, it gives you the ability to crop images on demand.
- Image widget crop: Provides the UI for content creators to be able to crop images on demand. It has a hard requirement of the Crop API module.
- Cropper: A simple jQuery image cropping plugin for added cropping functionality.
Note: With the exception of the Crop API module, the other two tools above are optional or can be replaced by other similar tools/modules. Often times the modules you pick for a job depend on your unique requirements. So don't feel you need to use the tools above if you know of other equivalent ones that may do a better job.
Next key steps to completeWith your tools inplace, you will probably need to complete the following steps:
- Creating crop types
- Create image styles that match your images rendering requirements (aspect ratio, dimentions, crop settings, etc)
- Adding a new effect to image styles that use "Manual crop"
- Create new responsive image style using the <picture> element
Crop types: They are in principle similar to image styles. They allow us to setup parameters for how to crop images. In particular, they help us predefine aspect ratios for cropping images. We can create a Square crop type, 3:2 crop type, 16:9 crop type, etc. Important to note, crop types's main job is to define cropping parameters in images, they don't restrict you from manually picking the image focal point.
Image styles: Since we are going to be doing art direction, we will need image styles that allow for custom/manual cropping. We already know how to create image styles, the ones for art direction will follow the behavior of the hero image on this page. Notice how the main images are wide when viewed in a large screen, but if you resize your browser or view the site on a mobile device the images become narrow and their aspect ratio changes. See both examples below.
The examples above shows an image cropped differently for different device sizes. This is Art Direction and is achieved using the <picture> element.
Responsive image styles using <picture>The biggest difference between creating responsive image styles for resolution switching and art direction is the "Breakpoint group" we select. Let's go through the process now:
- If you haven't already, enabled the Responsive image module
- Head over to /admin/config/media/responsive-image-style to begin creating a responsive image style
- Click Add responsive image style
- Type Banner as the label for the responsive image style. This name is solely based on the image examples above. Feel free to assign any name that makes sense to you.
- Select Olivero from the Breakpoint group dropdown. If you are using a different theme, choose that theme from the dropdown. We will discuss breakpoints in a bit.
- Scroll down and select a Fallback image style. Pick an image style that could be used on desktop and mobile if all failed with our responsive image style.
Breakpoints are defined in your theme's <theme-name>.breakpoints.yml which is located in the root of your theme. For each device you wish to target you create a breakpoint. Here's an example (I modified the breakpoints slightly for better read):
olivero.sm: label: Small mediaQuery: 'all and (min-width: 500px)' weight: 0 multipliers: - 1x - 2x olivero.md: label: Medium mediaQuery: 'all and (min-width: 960px)' weight: 1 multipliers: - 1x olivero.lg: label: Large mediaQuery: 'all and (min-width: 1440px)' weight: 2 multipliers: - 1xBased on the breakpoints found in *.breakpoints.yml, you should see an image similar to the one below which should reflect the same breakpoints.
- Expand any of the breakpoints fieldset and you should see a familiar set of settings. Think of each fieldset you expand here as the <source> element inside the <picture> element. This is where we choose a list of image styles. Then we repeat the process for each breakpoint. This is where we are telling the browser which images to use for each device size and therefore setting up art direction.
- The sizes field works the same as what we did in resolution switching.
- Once you have completed all yoru configuration, save your changes.
Once the responsive image style above is ready to go, You can repeat the process of creating a new view mode for your Media image and configuring it to use the new responsive image style you created above (Banner).
Navigate posts within this series
Mario Hernandez: Responsive images and Media
Drupal's Media is a powerful tool and we will use it to improve the way we manage responsive images by using media types and view modes to our advantage when managing responsive images. The following quote comes straight from Drupal's Media module:
In Drupal, the core Media module manages the creation, editing, deletion, settings, and display of media entities. Media items are typically images, documents, slideshows, YouTube videos, tweets, Instagram photos, etc. Media entities are standard Drupal content entities. And are grouped by Media type bundles. Like content types they can have fields added, and their display can be managed via view modes.
Media in DrupalOut of the box, Drupal ships with the following media types: Audio, Document, Image, Remote video, and Video. You can create new media types if needed. For example, you could create a new media type that is of type "Image". Why would you do this? Well, in some cases, you may have specific requirements to handle images of a particular entity and rather than modify the existing Image media type, you could create a new one so you can manage specific settings on the new type and don't run the risk of breaking something on your site by updating the default Image media type. Creating new media types is outside the scope of this post, but wanted to let you know what's available to you.
- Let's start by enableing the Media and Media Library modules as these are not enabled by default
- Navigate to /admin/structure/media
- Since we'll be working with images, click the Edit link to the right of the Image media type
- We won't get into all the wonders of Media types, instead let's jump into Manage display
Like any other Drupal entity bundle, Manage displays, also known as View modes, are a pretty powerful feature. So what are view modes? View modes is how we display content in different ways. Take a look at the image below. Each of the numbered marks is a view mode. Same content, different ways and different fields.
Media view modesNow that we understand how view modes work, we will use them to manage/display our responsive images. Let's start by creating a new Media view mode.
- Within the Manage display tab for the Image media type, scroll down and expand the Custom display settings fieldset
- Click Manage view modes
- In the View modes screen scroll down to the Media section and click Add new Media view mode
- Type News listing as the view mode name.
- Click Save
- Go back to the Image media type Manage display screen (/admin/structure/media/manage/image/display)
- Scroll down and expand the Custom display settings fieldset
- Check News listing and click Save. This enables the newly created view mode. You should now see it under the Manage display tab for the Image media type.
It's time to link the new view mode for the Image media type with the responsive image style we created.
- Withing the Manage display screen, click the News listing view mode we just created
- Change the Format of the image field to Responsive image. This allows us to pick a responsive image style of our choice.
- Click the gear icon to the right of the image field
- Under Responsive image style select Content grid. This is the responsive image style we created in the previous post.
- Link image to nothing
- Click Update
- Scroll to the bottom of the page and click Save
We just put all the pieces together which will result in well-managed responsive images. The order we followed is:
- Image styles: These are several image templates that share similar attributes (i.e. aspect ratio).
- Responsive image style: This is a collection of image styles that fullfill a specific use case.
- Media type and view mode: We have specific requirements for news and event article images. View modes allow us to fullfill these requirements without affecting other image displays.
So image styles, responsive image styles and image view mode are all connected and working great (you'll have to take my word for it 😃). One thing though, what content in our site will take advantage of all the beutiful work we've done? How about news articles? Take another look at the screenshot below. That's what we will do with all the work we've done.
Configure the Article content typeSince we opted to use news article images as the content to work with, there is a little clean up we need to do first in the Article content type.
The image field that ships with the Article content type does not use Drupal's Media, it's a basic image field. This does not mean we can't use responsive images with this type of image field, we can, but we get more out of the Media type field (view modes for starters). In addition, with the media type image field, we can take advantage of media library, add more fields to the image media type, reuse images, and much more.
Remove the image field from Article content typeWarning: Removing the image field will result in loosing any images you may have uploaded using this field. Only do this if you are sure you don't need the images. I never tested deleting the image field on a site that already had images so I am not sure if Drupal will even let you delete it.
- From the admin toolbar, click Structure, Content types
- Click Manage fields to the right of the Article content type
- For the image field, click the dropdown under Operations and select Delete
- Click Delete again to confirm you want to delete the image field. The image field is now gone
- Still within the Manage fields for the Article content type
- Click Create a new field
- In the Add a new field dropdown, select Media which is located under the Reference section. Notice there is also an image field, but this is the same kind we just deleted. Media is what we want.
- Type Image as the label for the new field
- Click Save and configure
- Keep the Allowed number of values as Limited to 1 and click Save field settings
- Optional but always a good practice is to add Help text. (i.e. Upload or select an image for this article)
- Check the box if you want to make this a required field
- Reference method should be Default
- Check the Create referenced entities if they don't already exist checkbox
- For Media type check Image (extremely important)
- Click Save settings. A new Image field is now available but this time it's a Media reference field of type Image.
By default the new image field was added at the bottom of the list of fields. Let's move the field up. We will follow the same steps for the Manage form display (for when content is created), and Manage display (for when content is displayed on the page).
- Within the Manage form display tab scroll down until you find the new Image field
- Drag it up so it displays right under the Title field
- Click Save
- Repeat for Manage display
- Still within the Article content type page, click Manage display
- Drupal by default provides 3 view modes for the Article content type: Default, RSS, Teaser, and Full content (not enabled by default). We can create as many new view modes as we want/need, but for this excersice we will use Teaser.
- Click Teaser from the list of view modes
- For the image field, make sure its Format is set to Rendered entity. Since the new Image field we added is of Media type, the image is an entity and we want to render it as such.
- Click the gear icon to the far right of the Image field
- Under View mode select News listing. This is the view mode we created for the Media image type field.
- Click Update then scroll down and click Save. That's it.
Before we can see the responsive images, let's create a couple of news articles so we have content to work with. Go ahead and create a few news articles using the Article content type. Upload large images if you wish so we can see the work we've done in action.
In the next post, we will complete the configuration and display of responsive images.
Navigate posts within this series
Mario Hernandez: Responsive image styles
In a nutshell, responsive image styles are a collection of image styles. It can be confusing because the similarities in their names, but responsive image styles are a bundle that holds one or more image styles.
What's the use of responsive image styles?If you read the posts about the <picture> element as well as the one about srcset and sizes attributes, we discussed that whether you are doing art direction or resolution switching, we need to provide the browser with a collection of images to serve to different devices. In Drupal the way we provide the collection of images is by using responsive image styles.
Naming responsive image stylesIn the previous post we went in detail about best practices for naming image styles. Properly naming responsive image styles is just as important but there are some differences in guidelines. While naming image styles is mostly based on the characteristics of the images (aspec ratio, orientation, dimensions), naming responsive image styles is typically based on their use case. Let's take a look at some examples.
Let's say we are building a photo gallery where we will use a series of images to display as a slider or photos grid. We may not be as concerned about the images aspect ratio or dimentions because we already have image styles in place to handle that. We're only interested on how the images will be used. In this example the use case is a Gallery. So a name for the responsive image style that makes sense to me would be Gallery or Photo gallery. Another example would be creating a responsive image style for your website's hero component. Regardless of what the images dimensions are, the responsive image style can be called Hero. Both of these examples are for very unique use cases, but there are also cases for more common type of responsive images styles such as a listing of news articles or events, a featured news article or a page for team member photos. In these cases, we can use names that we can reuse elsewhere. Something like Person listing, News full, Content grid, or 16:9 (Max 460px).
Back to hands-on exercisesWe are going to create a new responsive image style in which we will make use of the image styles we created in the previous post. Let's pretend the images we are targeting will be used for displaying News and Events listings (Similar to this, and example below).
Most websites use patterns that they repeat across their sections or pages. The news listing example above is a pattern that can be used for not only displying news articles, but maybe also events, social media posts, and more. This is great because identifying these patterns can help us create better image styles and responsive image styles that are reusable. This is what I meant in the previous post when I said that if you are fortunate enough to have designs for your project, you will be able to identify these patterns right away. Let's keep this in mind as we get back into exercise mode.
Resolution switchingHere's where all of our knowledge about <picture> (art direction) and srcset and sizes (resolution switching) comes in handy. We'll start with resolution switching because art direction requires additional tooling and configuration.
In Drupal 8 we used the Picture and Breakpoints contrib modules to handle responsive images. Starting with Drupal 9, Drupal provides the "Responsive image" core module which means we don't need to install any contrib modules. Responsive image is not enabled by default.
- Enable the Responsive image core module (/admin/modules)
- Once enabled, head over to /admin/config/media/responsive-image-style to begin creating our first responsive image style
- Click Add responsive image style
- Type Content grid as the label for the responsive image style
- Select *Responsive image from the Breakpoint group dropdown
- Scroll down and select a Fallback image style (16:9 (Max 320px))
- Expand the 1x Viewport Sizing [] fieldset
- Under Type, chose Select multiple image styles and use the sizes attribute
- Under Sizes type the following: (max-width:640px) 100vw, 30vw (I'll explain shortly)
- Under Image styles select the image styles we created before
- Scroll down and click Save
Since we are doing resolution switching and not art direction, we chose Responsive image from the Breakpoint group dropdown. Doing so presents to us the 1x Vieport Sizing [] screen with the following options:
- Type: Again, since we are doing resolution switching, the obvious choice here is Select multiple image styles and use the sizes attribute. The other two options are irrelevant in this particular example.
- Sizes: The Sizes option is where we tell the browser how big/small our images should be rendered in relation to the viewport being used. Depending on our goal, this field accepts a single value or a media query with some conditions. Let's say we wanted our images to always render at full width regardless of the device being used (like a Hero image for example), then the value for the Sizes field would be 100vw (100% the viewport width). In our case however, we want the image to display full width, but only if the viewport/device is not larger than 640px, otherwise, meaning if the viewport/device is larger than 640px, we want the image to display at 30% the viewport width. We could had also used a pixel-based value such as 400px for example.
- Image styles: This is where we choose the image styles we want to make available for the browser to pick from. Think of this as the srcset attribute in the <img> tag.
- Fallback image: We pick a fallback image in case all the above fails.
Very important: Remember in the Image resolution switching using srcset and sizes attributes post, our claim was that it's better to let the browser pick the best image possible as the browser is smarter than us? This is exactly what we are doing here. We are providing the browser with a collection of images to pick from. Each image provides its dimensions. Then we tell the browser how big/small we want the images to be rendered. One thing we are not doing is telling the browser which image to use, we let the browser make that decision. This is the complete opposite of what we do when we use <picture>. As a reminder, the method used above is what you would do in most of your images. This is like the default configuration for your responsive images. Only when you need to crop your images differently for each device size is when you would use the art direction approach.
Navigate posts within this series
Mario Hernandez: Image styles in Drupal
Now that we've gone over some very important concepts of responsive images (art direction and resolution switching), it's time to transfer all that knowledge and put it all in practice in Drupal. One of the key pieces for achieving responsive images in Drupal is by using image styles. Image styles are how Drupal manages the way we crop and size images.
What are image styles?Before we get to building image styles, let's go over what they are. Think of image styles as templates for cropping, scaling, converting, and sizing images. You can use these templates with any image on your site and as many times as you'd like. Thanks to image styles images will always render within the parameters we define.
A more real-world way of looking at image styles may be if you could imagine for a moment you have a couple of picture frames you'd like to use to hang some pictures in your house or office. One frame is 5x7, another is 4x6 and one last one is 8x10. The picture frames are Drupal's image styles.
So we have some picture frames and we have ordered several pictures from our favorite online photo printing service. There is one picture in particular I really love and I want to frame it using the 3 picture frames. So, although it is the same picture, I ordered different sizes of it (one for each frame size), and this will allow me to hang the 3 pictures in different sizes, aspect ratios and orientation. That in a nutshell are image styles.
Image styles best practicesImage styles are actually pretty easy to create but unfortunately because of this they can be misused or mismanaged. If not done properly, you may end up with a lot more image styles than you really need or image styles that are not well define and do no provide the outcome you are looking for. To avoid this, let's go over best practices for creating image styles which will result in less image styles to manage. Mind you, these are my best practices but I have to admit, they have worked very well for me.
Naming image stylesHave you heard the phrase "naming things is hard"? It's true. Unfortunately when it comes to image styles, if not named properly you can get yourself in a lot of trouble. Quick example, let's say I want to create an image style that I'd like to use on images of news articles that are displayed on the homepage. One might think a good name for the image style may be something like "Homepage news article images". It doesn't seem so bad but let me point out a few issues with this name:
- The image style is limited to the homepage
- It is limited to news article images
- It lacks information about the image dimensions or aspect ratio
One objective with image styles is to create them in a way that they are reusable. The more reusable an image style is the less image styles you will need to create which in turn becomes easier to manage. The main issue with the image style above ("Homepage news article images"), besides the 3 bullet points we called out, is that is not reusable. The name of it limits us to only use it on the homepage and only for news article images. If we want to display similar images elsewhere, we would need to create another image style maybe with the same parameters as the first one. You may be asking yourself, wait, why can't we use the same image style elsewhere? Technically you can, but think about how confusing it will be to use an image style called "Homepage news article images", not on the homepage and not on news article images.
Creating reusable image stylesOne very efficient way for creating reusable image styles is to name them based on the image aspect ratio or dimensions, or a combination of both. For example: "16:9 (Max 320px)", or "Box 1:1 (500px)". Here are some reasons why this is a great way to name image styles:
- They are not specific to any page or type of image (articles, events, etc.)
- They provide key information about the image aspect ratio and their dimensions
- I can use these image styles hundreds of times on any image that fits the requirements as well as on any page
- By creating/naming image styles this way, I may have just saved myself from creating many other image styles
I have found one of the most effective ways for identifyiing the image styles you need to create is by looking at your website mockups (if you are fortunate enough to have them). This may not always be possible, but if you do have designs for your website, this will tell you exactly which images you will need and how they need to be rendered. Having this information upfront will help you tremendously when creating image styles because you can plan ahead of time how to create reusable image styles that share commom attributes.
Image styles use casesWhen naming image styles it helps me to think of the characteristics of the images I am creating image styles for. For example, I have an image that should be rendered in 16:9 aspect ratio and it should not exceed a width of 320px. This is how I arrived at the name 16:9 (Max 320px). This also makes it possible to know which image style to use if I have other images that need to be rendered similarly. By the way, it is perfectly okay to use an image style that is slightly off from what an image needs to be rendered at. For example, Let's say I have an image that should be rendered at 16:9 aspect ratio, but its size should not exceed 250px. for this image, I can still use the 16:9 (Max 320px) image style.
A 100px or even 200px difference between the image style dimensions and the image you need to use it on it's an acceptable thing to do for a couple of reasons:
- 100 or 200px in most cases will not make a big of an impact in performance, however, if you are rendering 50 of these images in a single page, then this could certainly present performance issues. So my rule is as long as this is a oneoff type of situation, I'm okay with doing this.
- Keep in mind that just because your image may be resized larger than it actually needs to be rendered, your image will still visually rendered at the right size as I would suppose it is inside a container that will be sized to the right rendering size, via CSS.
- Being able to reuse an image style that may be slightly larger than needed saves me from creating more image styles.
I hope you see the impact good names for image styles have on your site. When you are working on an enterprise level website, using the best practices above can really help you with the maintenance of your image styles.
Image styles effectsEffects are the rules you set on each image style. Rules such as cropping, sizing, converting, saturating, rotating, and scaling of images is how we determine how to render the images in our site. In most cases, you want to let content creators of your site upload images that are relatively big. Doing so will allow you to use the images in your library in any use case. It is perfectly okay to scale your images down thorugh the use of image styles, but it is not recommended to scale images up. Doing so will result in blurry or pixelated images. This is why is better to upload large images. But you may be thinking, if I upload super large images, this will affect the performance of my site. It will if you are rendering the original images, but since we are using image styles, Drupal uses the original image to make copies at the size and aspect ratio you defined in your image styles. This is why by uploading a single image you are able to use it in many use cases at different sizes or aspect ratios.
Image styles effects can vary from image style to image style. For example, some image styles will require images to be scaled down, then cropped. Others will require images to be resized then cropped and others may just need for images to be resized to specific size. All these actions are called "Effects" in image styles. The effects you apply to your image styles will depend on the end goal for rendering the images. Let's do a quick demo of creating one image styles then applying effects to it.
Hands-on excerciseAll the principles covered in this series apply to Drupal 8, 9, and 10. You can follow along or simply watch me do it. To get started, spin up a vanilla Drupal site of the version of your choice. At the begining of this series I mentioned we will stick with only core/out of the box functionality to keep things simple.
Creating a new image style- Login to Drupal as administrator
- In your Drupal site navigate to /admin/config/media/image-styles
- Click Add image style
- For Image style name type: 16:9 (Max 320px)
- To keep things nifty, edit the image style machine name so it reads 16_9_max_320px. (Remove the trailing underscore generated by the ending parenthesis in the image style name. Clean machine names are important to me 😃. It actually makes a difference when you are debugging issues and you find your machine name has an extra underscore you didn't catch).
- Click Create new style
The image style above follows the best practices for name we covered earlier. This makes this image style reusable on any image that meets the aspect ratio and dimension requirements.
Adding effects to the image styleFor the purpose of this exercise, we are going to use the Scale and crop effect. This is probably the effect I use the most because it does exactly what I want, scale the image down to the size I want and crop it in the aspect ratio I need.
- While in the page where the new image style was created (/admin/config/media/image-styles/manage/16_9_max_320px), scroll down and you should see the Effect dropdown
- Select Scale and crop from the dropdown
- Click Add. The Add Scale and Crop effect screen will come up
- Type 320 for width and 180 for height. Note: These two values are required when you select the scale and crop effect. In other effects, these values may not always be required. It is important to define fixed dimensions on your image styles. This ensures your images will be sized/cropped at exactly the size you expect them to. How did I figure out the height for a 16:9 image with a width of 320px is 180px? I used this online aspect ratio calculator.
- Notice how you can change the focal point of the cropping by clicking any of the circles under Anchor. For this example we'll keep it in the middle circle.
- Click Update effect. This will bring you back to the image style page.
- We're done!
Now we have one custom image style with specific effects. If you noticed, Drupal comes with a couple of predefined image styles. If they work for your images you should make use of them. In this series we will be creating custom image styles.
As I mentioned earlier, names of image styles should be descriptive but not limiting. In the case of some of the image styles that come out of the box with Drupal, the names Large, Medium, and Wide do not seem like good choices for names because those names are all relative. Large in relation to what? Medium in relation to what? ...just sayin'.
Image multipliersOne thing we have not discussed but it is very important in responsive images, is the use of "Image Multipliers". What are image multipliers? In responsive images, you often think of image sizes in natural dimensions. If I think of an image that needs to be rendered at 720px, I will most likely resize or crop that image at 720px which makes total sense. With the evolution of high density or high resolution (retina) screens on mobile and larger devices, because they contain thousands or millions more pixels than traditional resolution screens, images need to actually be bigger than the intended size so they are rendered at their highest resolution/quality. This is what image multipliers are.
If we go back to the example above of the 720px image. For this image to be rendered as sharp and high-quality as possible in retina screen devices, we should add a 2x or 3x multiplier to it. Meaning, we should create an image styles for this image at twice and three times the intended size (1440px, 2160px). When we do this, the image will still be rendered at 720px (visually), but because we are providing larger images, these images will have twice or three times the number of pixels within them and when viewed in high resolution screens, the image quality will be superior than if we are only providing a regular 720px image. Note: I typically only create a 2x multiplier image styles for most of my images, but there may be situation when there are specific requirements for creating a 3x multiplier image styles.
Create a 2x multiplier image styleThe same way you created the original image style above for 16:9 (Max 320px), go ahead and repeat the process but this time create a 2x multiplier image style, 16:9 (Max 640px). Remember, the dimensions of this image style should be 640 x 360px.
So what's next?With our custom image styles in place, we can now make use of them, but before we do, let's go over another very important concept within Drupal, Responsive image styles ...whaaaaatttt?
We'll comeback to Drupal in a bit but first, we'll talk about responsive image styles in the next post.
Navigate posts within this series
Mario Hernandez: Image resolution switching using srcset and sizes attributes
In the previous article we defined what art direction is and how to address it using the <picture> element. In this post, the focus will be how to address responsive images when the requirement is image resolution switching. Resolution switching, in the context of responsive images, is rendering identical image content on all devices. Unlike art direction where each device gets a differently cropped image that may vary on aspect ratio, resolution switching uses images that are simply larger or smaller based on the device but retain the same aspect ratio and cropping settings. Resolution switching is how most images are rendered (the rule), the <picture> element approach is the exception to the rule. Take a look at an example of resolution switching below.
The image above demonstrate how multiple resolutions of the same image can be served to different devices. All the images in the example above are cropped exactly the same maintaining the same aspect ratio from large to small.
Using srcset and sizes attributesUsing the srcset and sizes image attributes is how most images are rendered in the web today. As indicated before, this is the recommended way for configuring responsive images if all you need is to switch resolution of images rather than art direction. So how does this approach work? Let's take a look at a typical configuration of the <img> tag using the image above as an example of the different image sizes we will want the browser to choose from:
<img srcset="original-image.jpg 2400w, extra-large.jpg 2000w, large.jpg 1600w, medium.jpg 1080w, small.jpg 800w, x-small.jpg 500w" sizes="100vw" src="large.jpg" alt="Image of sky shown at different resolutions" />Let's break things down so we can understand this approach better.
- <img>: Right off the bat we start by using a widely supported html tag.
- srcset: The srcset attribute in the img tag serves two important roles, 1) It stores a list of images that can be used by the browser, 2) Each image provides its width value which plays a role on the browser choosing the right image.
- sizes: The sizes attribute tells the browser the width, in relation to the viewport, the image should be rendered at. The value of 100vw shown above, means the image will be rendered at 100% the viewport width on all the devices. You could also use media queries like (max-width: 720px) 100vw, 50vw. This means that if the device does not exceed 720px in width, the image will be rendered at 100% the viewport width, otherwise (if the device is larger than 720px), the image will be rendered at 50% the viewport width.
- src: The src attribute is used as a fallback if everything fails.
Let me explain things in more detail because it is important we understand how this approach is so much better than using the <picture> element.
The biggest difference/advantage of using srcset and sizes versus <picture>, is the fact that we let the browser decide which image is the best image to render on any device. This is possible thanks to all the information we have supplied to the browser. For example, in the srcset we are not only providing the browser with a list of images to choose from, but we are also telling the browser how big each image is. This is very important because the browser will use this information when choosing the image to render. In the <picture> element approach, the image size descriptors are not available.
The sizes value tells the browser the size the image needs to be rendered at in relation to the viewport. This too is extremely important information we are providing the browser because if the browser knows the dimensions of all the images to choose from and how big/small the image needs to be rendered, then the browser is able to pick the best image possible.
But that's not all, the browser is smarter and knows more about the web environment than we do when a page or image is rendered. For example, the browser knows the viewport width used when viewing a website, it knows how fast/slow your internet connection is, and it knows about any browser preference settings (if any), setup by the user. Using all this information the browser is able to determine which image from the srcset is the best to use. In contrast, with the <picture> element, we tell the browser which image to use solely based on the device size.
Closing the gapNow let's see how using the srcset and sizes attributes closes the gap we identified when using the <picture> tag.
Environment conditions What the developer knowsduring development What the browser knows
during image rendering Viewport dimensions No Yes Image size relative to the viewport Yes No Yes via sizes Screen density No Yes Images dimensions Yes No Yes via srcset
Pretty nice huh? Now thanks to the srcset and sizes attributes we've closed the gap and the browser has all the information it needs to ensure the best image is served to each device.
The next post of this series will focus on image styles. These are fun but can also get you in a lot of trouble if not properly done. See you there.
In closingTime for a story: I recently did an experiment that 100% proves the use of resolution switching using srcset and sizes attributes. As most people nowadays, I use a very large second display when working on projects to fit more apps and see things better. My second display is nice but it's not a 4K display. It's double the physical size of my mac's screen, but the mac's screen resolution is higher by almost double (twice the number of pixels). When I look at an image of a project where I've implemented the practices in this guide, in the large display, and inspected the page, I see the browser has selected an image that is 720px which makes complete sense for the use case I am testing. I then unplugged the second display and viewed the page on my mac's screen (higher resolution), I reloaded the page and inspected it, I noticed the browser has now selected an image that is double the size of the first image. This is exactly the behavior I would expect because my mac screen is of higher resolution and my connection speed is very fast. So the browser was able to make the smart decision to use a different images based on my environment.
Navigate posts within this series
Mario Hernandez: Art Direction using the picture HTML element
In the previous article of this guide we covered the concept of responsive images and some of the challenges that come with implementing an effective system for them. In this article we will go in detail about the concept of "Art Direction" and how this applies to responsive images.
What is art direction?In the context of responsive images, art direction is the ability to display differently-cropped images based on the device size. For example, a large landscape shot of a person rowing in the middle of a lake is shown when viewed on a large desktop device. If we were to use the same image on a mobile device, that image would shrunk down, making the person in the image very small and hard to see. A better option would be to show a different version of the image that zooms in and focuses on the most important part of the image, the person rowing. See an example of this image below.
Enter the <picture> HTML elementIn order to achieve art direction we need to be able to query for the size of the device being used to view the website. Once we've identified the device size we instruct the browser which image to use based on the device size. This will allow us to provide a better user experience as each device will display an image intended specifically for that device. Going back to the image above, we can see that the main image has been cropped differently to ensure the most important part of the image is displayed on each divice.
So how do we query for the device size and how do we instruct the browser which image to use? This is where the <picture> element/tag comes in. Let's take a look at the code that makes all this possible and break it down.
<picture> <source media="(min-width: 2400px)" srcset="images/rowing-2400.jpg 1x, images/rowing-4800.jpg 2x" type="image/webp"> <source media="(min-width: 1280px)" srcset="images/rowing-1400.jpg 1x, images/rowing-2800.jpg 2x" type="image/webp"> <source media="(min-width: 640px) and (max-width: 1279px)" srcset="images/rowing-1200.jpg 1x, images/rowing-2400.jpg 2x" type="images/webp"> <img src="images/rowing-1200.jpg" srcset="images/rowing-2400.jpg 2x" alt="Person rowing on a lake" width="1200" height="800"> </picture>Note: The order in which the media queries are written within the <picture> tag matters. The browser will use the first match it finds even if it's not the intended one. Therefore, consider the media query order very carefully to ensure the right image is served.
- <picture>: The <picture> tag is simply a wrapper. On its own it does not do anything.
- <source>: The <picture> HTML element contains zero or more <source> elements. The browser will consider each child <source> element and choose the best match among them. If no matches are found—or the browser doesn't support the <picture> element—the URL of the <img> element's src attribute is selected. The selected image is then presented in the space occupied by the <img> element.
- Within the <source> element, you will find some very handy attributes (media, srcset, and type):
- media: Rembember earlier we said we need to query for the device size? Well, within the media attribute you can write media queries much like the media queries you write in CSS (media="(min-width: 600px)"). This is how we check the size of the device when a page is rendered.
- srcset: This attribute allows us to provide a list of images the browser can use when the media query finds a match (srcset="img-768.jpg, img-1440.jpg").
- type: The type attribute specifies a MIME type for the resource URL(s). This is optional if using common image types such as JPG, PNG, TIFF, etc. If you plan on providing images in different file formats, you can do so using the type attribute. This is handy in the event the browser does not support a specific file type (type="image/avif"), as you can then provide a supported file type.
- <img>: The img element serves two purposes:
- It describes the dimensions of the image and its presentation
- It provides a fallback in case none of the offered <source> elements are able to provide a usable image.
And there you have it. The <picture> element is a great way to serve different images based on things like device size or screen density. When the <picture> element was first introduced it required a pollyfill as not all browsers supported it. Nowadays, unless you are supporting Internet Explorer 11 (sorry bro), all other major browsers provide native support for it. Take a look at the chart below for current browser support.
Great! Let's use <picture> on all our images ...NOOOOOOOO!!!!!!Say what? If the <picture> element is so great, why can't we use it for rendering all of our images? Well, as great as the <picture> element is, it should not be the default solution for serving responsive images in your site. The only use case for the <picture> element is when you are trying to achieve "Art Direction" (cropping your images differently for each device size).
Remember at the begining of this post when I said "In order to achieve art direction we need to be able to query for the device size. Once we've identified the device size we instruct the browser which image to use..."? There lies the problem. Let me explain.
The issue with the statement above is that "we are telling the browser which image". Not only that, but we are doing so solely based on the size of the device. This may not always be the best way to determine which image a device should use. Imagine you are using a nice relatively new laptop with a super high density screen. Based on our rules established within the <picture> element code snippet above, we would end up with an image that is 4800px in size. This is a pretty large image but it's the one that meets our creteria defined in the media query above. If you're home with a decent wifi connection you will never see any issue loading an image this large, but imagine you are working out of a coffee shop, or at a conference with poor wifi connection, or worse yet, you're on the road using your phone as a hotspot and your signal is very bad, now you will really experience some performance issues because we are telling the browser to load the largest image possible because your computer screen is big (relatively speaking). With the <picture> element we can't check how fast your internet connection is, or whether there are browser preferences a user has configured to account for slow internet speeds. We are basing everything on the size of the device.
Then why use the picture element? Well, when developing a website, the developer does not have all the information they need to serve the best image. Likewise, when rendering a page and using the <picture> tag, the browser does not know everything about the environment. The table below shows this in more detail and exposes the gap between the developer and the browser.
Identifying the gap when using <picture> Environment conditions What the developer knowsduring development What the browser knows
during image rendering Viewport dimensions No Yes Image size relative to the viewport Yes No Screen density No Yes Images dimensions Yes No
You may be wondering: "Why did you get us all excited about the <picture> element if we can't really use it?" well, if you are trying to achieve art direction, then you use the <picture> element. It's the recommended approach for that use case. If you are looking for resolution switching, a use case for most images in the web, you need to use the srcset and sizes attributes approach. In the next post we'll dive deep into this technique.
Navigate posts within this series
Mario Hernandez: Responsive images in Drupal - a guide
Images are an essential part of a website. They enhance the appeal of the site and make the user experience a more pleasant one. The challenge is finding the balance between enhancing the look of your website through the use of images and not jeopardizing performance. In this guide, we'll dig deep into how to find that balance by going over knowledge, techniques and practices that will provide you with a solid understanding of the best way to serve images to your visitors using the latest technologies and taking advantage of the advances of web browsers in recent years.
Hi, I hope you are ready to dig into responsive images. This is a seven-part guide that will cover everything you need to know about responsive images and how to manage them in a Drupal site. Although the excercises in this guide are Drupal-specific, the core principles of responsive images apply to any platform you use to build your sites.
Where do we start?Choosing Drupal as your CMS is a great place to start. Drupal has always been ahead of the game when it comes to managing images by providing features such as image compression, image styles, responsive images styles and media library to mention a few. All these features, and more, come out of the box in Drupal. In fact, most of what we will cover in this guide will be solely out of the box Drupal features. We may touch on third party or contrib techniques or tools but only to let you know what's available not as a hard requirement for managing images in Drupal.
It is important to become well-versed with the tools available in Drupal for managing images. Only then you will be able to make the most of those tools. Don't worry though, this guide will provide you with a lot of knowledge about all the pieces that take part in building a solid system for managing and serving responsive images.
Let's start by breaking down the topics this guide will cover:
- What are responsive images?
- Art Direction using the <picture> HTML element
- Image resolution switching using srcset and sizes attributes
- Image styles and Responsive image styles in Drupal
- Responsive images and Media
- Responsive images, wrapping up
A responsive image is one whose dimensions adjust to changes in screen resolutions. The concept of responsive images is one that developers and designers have been strugling with ever since Ethan Marcotte published his famous blog post, Responsive Web Design, back in 2010 followed by his book of the same title. The concept itself is pretty straight forward, serve the right image to any device type based on various factors such as screen resolution, internet speed, device orientation, viewport size, and others. The technique for achieving this concept is not as easy. I can honestly say that over 10 years after reponsive images were introduced, we are still trying to figure out the best way to render images that are responsive. Read more about responsive images.
So if the concept of responsive images is so simple, why don't we have one standard for effectively implementing it? Well, images are complicated. They bring with them all sorts of issues that can negatively impact a website if not properly handled. Some of these issues include: Resolution, file size or weight, file type, bandwidth demands, browser support, and more.
Some of these issues have been resolved by fast internet speeds available nowadays, better browser support for file tyes such as webp, as well as excellent image compression technologies. However, there are still some issues that will probably never go away and that's what makes this topic so complicated. One issue in particular is using poorly compressed images that are extremely big in file size. Unfortunately often times this is at the hands of people who lack the knowledge of creating images that are light in weight and properly compressed. So it's up to us, developers, to anticipate the problems and proactively address them.
Ways to improve image files for your websiteIf you are responsible for creating or working with images in an image editor such as Photoshop, Illustrator, GIMP, and others, you have great tools at your disposal to ensure your images are optimized and sized properly. You can play around with the image quality scale as you export your images and ensure they are not bigger than they need to be. There are many other tools that can help you with compression. One little tool I've been using for years is this little app called ImageOptim, which allows you to drop in your images in it and it compresses them saving you some file size and improving compression.
Depending on your requirements and environment, you could also look at using different file types for your images. One highly recommended image type is webp. With the ability to do lossless and lossy compression, webp provides significant improvements in file sizes while still maintaining your images high quality. The browser support for webp is excellent as it is supported by all major browsers, but do some research prior to start using it as there are some hosting platforms that do not support webp.
To give you an example of how good webp is, the image in the header of this blog post was originally exported from Photoshop as a .JPG, which resulted in a 317KB file size. This is not bad at all, but then I ran the image through the ImageOptim app and the file size was reduced to 120KB. That's a 62% file size reduction. Then I exported the same image from Photoshop but this time in .webp format and the file size became 93KB. That's 71% in file size reduction compared to the original JPG version.
A must have CSS rule in your projectBy now it should be clear that the goal for serving images on any website is doing it by using the responsive images approach. The way you implement responsive images on your site may vary depending on your platform, available tools, and skillset. Regardless, the following CSS rule should always be available within your project base CSS styles and should apply to all images on your site:
img { display: block; max-width: 100%; }Easy right? That's it, we're done 😃
The CSS rule above will in fact make your images responsive (images will automatically adapt to the width of their containers/viewport). This rule should be added to your website's base styles so every image in your website becomes responsive by default. However, this should not be the extend of your responsive images solution. Although your images will be responsive with the CSS rule above, this does not address image compression nor optimization and this will result in performance issues if you are dealing with extremly large file sizes. Take a look at this example where the rule above is being used. Resize your browser to any width including super small to simulate a mobile device. Notice how the image automatically adapts to the width of the browser. Here's the problem though, the image in this example measures 5760x3840 pixels and it weights 6.7 MB. This means, even if your browser width is super narrow, and the image is resized to a very small visual size, you are still loading an image that is 6.7 MB in weight. No good 👎
In the next post of this series, we will begin the process of implementing a solution for handling responsive images the right way.
Navigate posts within this series
Mario Hernandez: Drupal base path
Recently I was building a component that required a static image which was not stored in the database but instead needed to be stored somewhere in the file system of the site. There are several ways for serving a static image for example we could have stored the image in the sites/default/files/images/ directory. A very common approach which in many cases would work just fine, however, in my case I was building a component and I wanted for the component image to be located within the same component's directory. This makes sense because if I wanted to reuse or share this component, all component assets would be included in a single directory.
RequirementsMy goal with this task was to dynamically point to the image regardless the site this component was running on. Oh yeah, we are running a multi-site architecture with hundreds of sites and a single code base. So this component needed to work in all of these hundreds of sites. Now the challenge seems a little more... "challenging".
¡Manos a la hobra'!'I started by doing research to determine the best way possible to achieve this. I read about using a pre-process function that would generate a dynamic base path of the site but I was hoping I could keep things simple and do everything on the front-end with only Twig. This would make it a more appealing approach for front-end developers.
After some research, I came across two little gems that became game-chargers for my project. One of these gems is the {{ url('<front>') }} Twig function. This will provide the current site's homepage/base path. The other very handy Twig function is {{ active_theme_path() }} which prints the path of the current active theme (themes/custom/my_theme). While researching for this task, I also found you can use the {{ directory }} Twig variable in your theme's templates to print the active theme's path. A word of coution when using either the {{ active_theme_path() }} function of the {{ directory }} variable as these could have different results depending on your whether you are using them in a base or sub theme. Here's a drupal.org issue that discusses this in more detail.
Armed with these two little functions, and one Twig variable, we can now work in generating a dynamic path to our theme's directory where the static image for our component is located. So this may seem like a simple thing but remember, our component's image should work regardless of the site the component is used on within our multi-site architecture. Some sites even use a different sub-theme but the parent theme is always the same which is where our image is stored.
Before we can use the first function we need to run it through the |render Twig filter. Since Twig will return an array from the {{ url() }} function, we need to convert it to a string because we need the value of the function. Let's take a look:
{{ url('<front>')|render }} # This will give us http://my-site.com/Next let's work with the theme path function. Similarly to the function above, we will use the |render Twig filter to convert it from an array to a string.
{{ active_theme_path()|render }} # This will give us themes/custom/my-themeNow that we have two strings we can joint them together to compose the full path to our image:
<img src="{{ url('<front>')|render }}{{ active_theme_path()|render }}/images/image.jpg" alt="alt text" />If we want to get fancy we could actually set a variable to shorten things a bit:
{% set theme_url = url('<front>')|render ~ active_theme_path()|render %} <img src="{{ theme_url ~ '/images/image.jpg' }}" alt="alt text" />And there you have it. A dynamic path that will work on any of our sites.
I realized some people reading this already knew all of this but I didn't. So I figured I would share it because I bet there are others out there that also do not know about the {{ url('<front>') }} or {{ active_theme_path() }} Twig functions as well as the {{ directory }} variable. As I said before, there are many ways to handle this challenge, but in my case this is exactly how I wanted to approachh it. I hope this was helpful. Cheers!