I'm thinkin you need to stay a bit longer.

JK
If you wait till the last part of April or first of May, the desert floor is typically on fire with poppies and other wildflowers. Everything is blooming and the weather is usually a little more tolerable. There is still a climb of over 6000 feet in elevation near Hole in the Wall. If you camp there, it will get pretty chilly over night.
Great run for history and all out relaxation. Drive slow, take a breath, enjoy the beauty only the Mojave can give, and relax.
The best time for me so far was a 3 night trip. We started in Laughlin, camped at Mid hills the first night, camped at 17 mile the second night, then at afton canyon campground the 3rd. Each day we set out by 9:30 am and we were setting up camp by 4:00. That gave us plenty of time to sit around a campfire with our drink of choice and truly relax.