Hmmm SXSW, I guess that be the kind of gig to expect press and label scouts. It most certainly sounds like this agent is testing you (your band).-- Are you (and your band) really that keen, Yes/No?
I've been in that situation repeatedly and yes it's can be freaking nightmare. --- Sydney (downtown city - central) in Australia, the traffic is so bad the roads are like a over-crowded parking lots ; most people are advise to leave their cars at home and take a flight, taxi, train or bus, BUT obviously for a band or courier vans/rucks that is not an option (imagine how they cope, think about that and you might come up with solutions).
I'm not entirely sure how business is done in the US, in Australia, bands are required to sign a contract (this is usually through major agents, at least 4 weeks in advance) so unless the entire band is hospitalized, you better 'all' show up.
Tip: (1) get to the city a day early, (2) book a room (unless the venue supplies 'one') yes, probably share a room together lol. (3) To-heck-with-it = parked in an awkward spot close to the venue, two guys directed traffic around and the others VERY quickly off-loaded the gear on the pathway... then load in whilst one person kept watch on the gear. In my case, as the band made enough money, we hire a road crew and luckily most venue bookings included all expenses paid - room, but I never stayed in those (others did) I prefer my own room at my own expense.
Yep, sounds like this agent is really testing you (and your band)! Once the money rolls in, hire a truck/driver for these types of gigs but always have a couple of band members arrive to keep an eye on load in/outs.
I wish you all the best with your endeavors. Good luck!