What a mess.
Cut your losses. Let them walk away and don't give them the stems. Only copies of the master should be given in which case it seems they didn't even get to that stage. All this should have been in the terms.
Masters? Studios can also charge extra per-copy (masters, not to be confused with mastering costs, hourly rate)!
In General - to whomever happens to stumble on this thread: Every professional recording studio I've been involved with charge by the hour and usually IIRC it's up-front deposit and then paid in full on the day just prior to the session, or in advance (optional)
For all bookings, copies of receipt/invoice along with cancelation policy must be included; outlining in clear terms that at least 48 hours notice of cancellations must be given should the client request a refund in full or for the deposit, or simply wishes to change the day and length (hours) of the session.
Most studios have a 'minimum hourly rate' for example, 4 hours for multi-track recordings, 1 hour for mastering. Some studios may offer discounts for longer sessions and/or well in advanced bookings.
Some studios may also be generous enough to allow up to 2 hours over-time to get the job done at no cost to the client. A courtesy gesture I guess LOL
In clear terms; setup time is at the expense of the client (included in the hourly rate)! That is why it's advised that clients have their act and their gear together before making a booking. I've had my fair share of experience over the years to know, NOT to run a pro rec studio business LOL!! There is just too much involved; opened for all types of bookings, long hours, staying up-to-date, cleaning, maintenance and being competitive are just some examples.
Standard or Advance Sessions: Some studios also charge more (hourly rate) for additional studio gear. A request that is not included in the standard booking. I've seen this arrangement done because sometimes gear requests for more expensive brands seem so extravagant and sometime not necessary but just to please a stubborn client LOL. But there is a workaround for the struggling studio and that is to hire gear from a third-party (pro gear hire) cost are weighed in the booking fee.
My experiences are based in Sydney, Australia. I have no idea if the offers and terms are exactly the same elsewhere. And I have not been involved in the business for a long time = things may have changed since.