I think the key is to start really slow and go at your own pace. Don’t try to rush it or try something you don’t feel ready for yet. This probably means learning with a friend or family member rather than an instructor because my understanding is the courses are often fast-paced (but I could be wrong).
For my first driving lesson, my dad and I went to the parking lot of my community college on Thanksgiving Day because that’s when we knew it would be completely empty. It was a huge parking lot so I didn’t have to worry about crashing into anything. I learned how to turn the car on, go into drive/neutral/reverse/park, drive, brake, turn, and turn the car off.
I got to go super super slowly at first because I wasn’t on a road so there was no one behind me to get impatient. I could brake as much as I wanted. Then once I started getting more confident, I went a little faster and worked on turning and getting a sense for how the car moved and where it was in space.
You can practice in a parking lot as many times as you want before trying streets, but once you do, you’ll want a quiet neighborhood without a lot of car traffic, where cars are likely to go slowly. The person teaching you can drive you out to somewhere like this if you don’t have it in your neighborhood. My dad took me to a neighborhood nearby like that and I practiced just driving slowly, making turns, and stopping and looking at intersections. To get to and from our house we needed to go on a scary fast road, so he drove on that.
Eventually he had me drive on the scary fast road, then bigger scary fast roads, and I started driving myself to classes every day on it (with my mom next to me) so I would get more and more confident. Eventually I was completely comfortable doing that but hadn’t driven on a highway yet. So he drove me out to a “smaller” chiller highway than the ones near us for me to practice on, then eventually I practiced on the interstate, and then finally the one Big Scary interstate.
This doesn’t need to happen fast. I had my first driving lesson in November 2012, and took my driving test in March 2014.
I have ADHD, and so do a lot of people who drive! It may turn out that driving isn’t something you are able to master, but I would not assume that just from being nervous at one session! I would encourage you to give it another try, and to go slowly — both in terms of progression and literally. If you want to spend your entire first lesson just driving in a straight line at 1 mile per hour, that is okay.