How to program a touchscreen on an ESP32

3/30/2024 | by Bits and Droids

In a previous article, we looked into adding visual elements to a generic TFT screen for our primary flight display. But what a waste it would be if we didn't utilize the screen fully. That is why we will bring our flight display to the next level today by adding touch support. This guide will work with screens driven by ILI9341, ILI9481, ST7789, and many more chips.

Because an Arduino struggles to drive a 4-inch LCD display, I'd highly recommend using a more powerful board like the Teensy, STM32, or, my favorite, the ESP32. The ESP32 has the perfect balance between price, performance, and features. This will also be the board we will use for this guide. This doesn't mean you can't use a different board, but if you do, there is a chance you need to find libraries tailored to your specific board.

Before we dive right in, we need to install 2 libraries:

TFT_eSPI (This library will drive our screen and handle the touch inputs)
BitsAndDroidsFlight32 (if you want to use the screen to send commands to Microsoft Flight Simulator 2020)

Used in this tutorial:

Touchscreen technologies

There are 3 main technologies used in the touchscreen industry. One of these is touchscreens based on sonic wave technology. When your finger touches the screen, you disrupt the sonic wave, giving the device away to recognize where you've touched the screen. But the 2 most well-known technologies are capacitive and resistive screens.

Capacitive screens

Capacitive screens can be found in almost every household. There is a high probability that you either read this article on a capacitive screen or have a capacitive screen in your pocket. A capacitive screen consists of a nonconductive material like glass covered by a transparent conductive layer. We, humans, are also electrically conductive. We disrupt the electric static field by touching the screen, which can be measured and used to determine where we've touched the screen. A simple way to recognize these screens is to touch them with a simple plastic stylus or a glove (not one of those special gloves with conductive fingertips). Nothing will happen if it's a capacitive screen since the non-conductive material doesn't disrupt the electric field.

Resistive screens

This technology is the cheapest way to create a touchscreen. It places 2 resistive layers on top of the screen, separated by air. When you touch the screen, the upper layer contacts the layer below. From here on out, it acts as a simple voltage divider (almost like a keypad). By checking the X and Y axes, we can cross-reference where the screen has been touched. Even though it's the cheapest technology, it doesn't make it bad. It can withstand liquids and other contaminants well, and you can touch the screen with any stylus. Our generic LCD screen also uses this technology.

Let's get started

Now that we know how it works, it's time to start coding. The first thing we need to do is include our libraries and create a TFT_eSPI object and a BitsAndDroidsFlight32 object. The TFT_eSPI class contains all the logic we need to display visual elements and register our touches. By creating an object, we can access all these functions that belong to the class by referencing our object.

cpp
#include <SPI.h> //This is the generic SPI library of the ESP32
#include <TFT_eSPI.h> //This is our lcd library
#include <BitsAndDroidsFlight32.h> //This library let's us receive and send data to MFS2020

//Here we create our TFT_eSPI object and name it tft
TFT_eSPI tft = TFT_eSPI();
//Here we create our BitsAndDroidsFlight32 library and name it connector
BitsAndDroidsFlight32 connector = BitsAndDroidsFlight32();

Drawing some elements

We need to add some visual elements to the press to see if our touchscreen logic works. If you've been following along with the radio display video, you can use those elements as a starting point. To keep things clean and tidy, I will be adding 2 empty buttons to demonstrate the logic used.

Buttons with rounded corners look more appealing than sharp corners. The TFT_eSPI library has a function called tft.fillRoundRect(x starting point, y starting point, width, height, corner radius, color). We could fill in these parameters to draw a rounded square. But to keep things tidy and clean, we will create a structure that holds all the data for our rounded square. We then add a function to draw these elements on our screen. Check out this article for a full, in-depth guide on how this works. check out this article.

cpp
byte xMargin = 30;
byte magin = 20;

// This struct holds all the variables we need to draw a rounded square
struct RoundedSquare {
  int xStart;
	int yStart;
	int width;
	int height;
	byte cornerRadius;
	uint_16t color;
};

RoundedSquare btnA = {
	xMargin,
	margin,
	150,
	50,
	4,
	TFT_WHITE
};

//btnB takes btnA as refference to position itself
RoundedSquare btnB = {
	btnA.xStart + btnA.width + margin,
	margin,
	150,
	50,
	4,
	TFT_WHITE
};

//This function will take a RoundedSquare struct and use these variables to display data
//It will save us more code the more elements we add
void drawRoundedSquare(RoundedSquare toDraw){
	tft.fillRoundRect(
		toDraw.xStart,
		toDraw.yStart,
		toDraw.width, 
		toDraw.height, 
		toDraw.cornerRadius,
		toDraw.color
	);
}

void setup(){
	Serial.begin(115200);
	tft.setRotation(1);
	tft.init();
	//Set our background to black
	tft.fillScreen(TFT_BLACK);
	//This passes our buttons and draws them on the screen when our arduino boots. 
	drawRoundedSquare(btnA);
	drawRoundedSquare(btnB);
}

Storing touch coordinates

We draw elements using a coordinate system. The top-left pixel is pixel X0, Y0 on our grid. To make our lives easy, our touch system uses the same coordinates. If we touch the top-left pixel, the returned coordinates will be X0 and Y0, respectively. There is a minor catch if you rotate the screen horizontally. The default orientation in the library is the portrait mode. When we rotate the screen, the touch coordinates can be flipped on the Y-axis. Our top row will be 320, while our bottom row is 0. There is probably a way to calibrate the screen in the library, but since the logic is quite simple, we will apply the flip ourselves. The formula for this is 0 + (maxWidth - the TouchY). If we touch the top-left pixel, the formula becomes 0 + (320 - 320) = 0. This matches the coordinate system we've used to create our buttons.

We need to store these coordinates somewhere, so we'll start by adding a variable for the X and Y coordinates.

cpp
BitsAndDroidsFlight32 connector = BitsAndDroidsFlight32();;
...

uint16_t t_x = 0, t_y = 0;

...

Check if the screen is touched

We could read the raw output of the screen, but some debouncing and smoothening can improve your experience by a mile. Luckily, the TFT_eSPI library does all the hard work for us. The function tft.getTouch(&t_x, &t_y) returns false if the touch is invalid. If the touch is valid, it stores the coordinates at our t_x and t_y coordinates.

To keep things simple, we will continuously poll our screen to check if our majestic fingers are pressing down on the device. But we want to keep the loop function tidy, so I'm going to wrap the logic in a new function called checkTouched().

cpp
//It's a void function since we don't return anything
//This function will mainly excecute the logic for the touch handling
void checkTouched(){
	//Remember this function returns false if the touch is invalid
	//C++ asserts something as true if it's not 0. So with a valid press
  //This statement will assert to true
	if(tft.getTouch(&t_x, &t_y){
		
  }
}

void loop(){
	checkTouched();
}

Addresses and pointers

If you code in C/C++ (especially with embedded systems like Arduino), you'll encounter the terms' pointers and addresses sooner or later. But Dave, why do you include this chapter here? Notice how the getTouch functions take the parameters &t_x and &t_y. By prefixing the variable with an &, we indicate that we do not pass that value stored at t_x and t_y but rather at the address. Each variable we create lives somewhere in the memory of our microprocessor. The mailman would have a horrible time if our house had no address. The same applies to our digital mailman.

Let's illustrate this in a small pseudo sketch.

cpp
//Here we create a simple int with the value 0
//This value is now stored at a specific address in our memory
int a = 0;

//This takes the value of a and copies it to a new variable called b stored at a new address
int b = a;

//The * indicates that we want to store a pointer of an int variable
//By prefixing the a with an & we indicate we want to store the address of a
int* c = &a;

//Now we create an int that points to the value stored at the address stored at c
//Since c contains the address of a this will set the value of d to the value stored at a
int d = *c;

Now, what happens with our touch coordinates? When we pass the address of our x and y variables, the library checks to see if the touch is valid. Instead of creating a new temporary variable that contains the touch coordinates, it takes the address we pass, checks if the touch is valid, and directly alters the value saved at our address.

Pointers and addresses can feel daunting when you're new to coding or if you're used to higher-level languages like Java or C#. But don't feel pressured that you have to use them in all your code. In many instances, they can be avoided and add another level of complexity. When you really want to memory optimize your code, these topics play a role. For now, I want you to know what these terms mean and how you can recognize them. This makes a line of code like &t_x a lot easier to dissect, even if you don't know what happens to the T.

Button boundaries

So far, we know if the screen is touched, but that doesn't get us far. We want to know if the button is pressed, and only if the button is pressed do we want to trigger an action. Since we know that the coordinate system of our elements matches the coordinate system of our touch inputs, we can draw a mental box around the button.

The press has to be at a higher X position than the X starting position of the button and lower than the right edge. The right edge can easily be obtained by taking the X starting position and adding the width. We can do the same for the Y position if the touch is higher than the Y starting position and lower than the Y starting position + the height of the button. We know it's on the same vertical row as the button. Combining both gives us a nice box shape around our buttons.

cpp
void checkTouched(){
	if(tft.getTouched(&t_x, &t_y){
		//Here we check if the touch is contained within our imaginary box
		if(t_x > btnA.xStart && t_x < btnA.xStart + width && t_y > btnA.yStart && t_y < btnA.yStart + height){
			connector.send(btnA.command);
		}
	}
}

We could hardcode each action we want to associate with each press, but including the action with the struct is easier. This way, we can change the action once, and when we call the btn action variable, the corresponding action will be executed.

cpp
struct RoundedSquare {
  int xStart;
	int yStart;
	int width;
	int height;
	byte cornerRadius;
	uint_16t color;
	//Here we add the command
	int command;
};

RoundedSquare btnA = {
	xMargin,
	margin,
	150,
	50,
	4,
	TFT_WHITE,
	//add the desired command (see the docs for all available actions)
	sendG1000PfdMenuButton
};

//btnB takes btnA as refference to position itself
RoundedSquare btnB = {
	btnA.xStart + btnA.width + margin,
	margin,
	150,
	50,
	4,
	TFT_WHITE,
	//add the desired command (see the docs for all available actions)
	sendG1000PfdFlightPlanButton
};

Do I have to repeat myself?

So now we check if button A is pressed but also have a button B. And perhaps you've got even more buttons on your screen than I have now. You could do two things. Create a check for every button, or find a way to loop through all your buttons at once. The first will give you a mountain of code that will be a nightmare to maintain. The latter will give you a cleaner and readable solution.

But how are we going to achieve this? The easiest way to loop through all our buttons is to store them in an array. We can then loop through each element in the array and perform our boundary check as we id with our button A.

cpp
RoundedSquare buttonArray[] = {btnA, btnB};

void checkTouched(){
	if(tft.getTouched(&t_x, &t_y){
		for(auto & button : buttonArray){
			if(t_x > button.xStart && t_x < button.xStart + button.width && t_y > button.yStart && t_y < button.yStart + button.height){
				connector.send(button.command);
			}
		}
	}
}

What the loop?

Perhaps you've often seen me use the plain old regular for loop. It goes something like this:

cpp
for(int i = 0; i < something; i++){//do something with buttonArray[i]}

This creates a loop with a counter that we use to access each element in our buttonArray stored at position i.

In C++, you can also access each element in an array using a different approach. It goes like this:

cpp
for(auto & button : buttonArray){//do something with button}

The word button is a name we give each element in the array. This can be anything your heart desires. We loop through each element in that array by referencing the word button. The word auto tells us that we want to detect the type of elements in the array automatically. In our case, the array is filled with data of the type RoundedSquare. This automatically makes each element of the type RoundedSquare as well. This doesn't give us a counter unless we add that logic ourselves.

Reusing buttons

Let's say you've got 10 buttons that you want to use on 2 screens. The action has to differ depending on the active screen. You could add 20 buttons and swap them out when you open a new screen. However, you could also swap out the command to work more efficiently when you change screens. This lets us store the physical settings like the width and height once while we swap out the command as needed.

cpp
int radioCommandArray[] = {sendCom1WholeInc, sendCom2WholeInc};
int g1000CommandArray[] = {sendG1000PfdMenuButton, sendG1000PfdFlightplanButton};

//This part of the code can be placed at the same spot where
//you initialize the respective screens
//I let it up to you where it would fit be
if(mode == RADIO){
	for(int i = 0; i < sizeof(radioCommandArray); i++){
		buttonArray[i].command = radioCommandArray[i];
	}
}
if(mode == G1000){
	for(int i = 0; i < sizeof(g1000CommandArray); i++){
		buttonArray[i].command = g1000CommandArray[i];
	}
}