jonboper
Hmm, some very interesting synths in this list.  Zampler is like a free Dimension Pro, kind of.  In all the years I've owned them (8 years?) I've never used Rapture or Dimension this way...the interfaces just don't lend themselves to sample manipulation in an intuitive way, all the sfz notepad-y typing, I looked into it when the synths came out but the learning curve seemed too steep.  I guess that's why I loved Iris so much when it came out, you can look at it and see what you're doing, for me a more fun and creative process.
 
I'll try out Rapture again, but the leaders from this list as far as what I'm looking for are TX16Wx and Geist, I think (I can't afford Geist right now, but it looks like a pretty capable synth).
 
Thanks for all the suggestions - you all put together a better list than any of the other forums I searched through with the same question...

As a geist owner, I can tell you firsthand that it's nothing like Iris. It is great at chopping up, resequencing samples and loop manipulation, but not going to be useful at creating lush sample based pads and such. It's more of a hip-hop/EDM and remixing beast. It does go on sale at times. 99 is typical and i got for $49 once. Occassionally a lite version pops up for free.
 
Iris is a very unique synth so getting anything close is a challenge. I can't  imagine Rapture or Dim Pro sounding anything like Iris, but I have never tried. Considering it is a free option, it wouldn't  hurt to try, but i suspect Alchemy will get you closer.

There is this free alchemy cm version which is kind of making me not want to buy the full version (although I do not want to buy any synth atm).

Huh, Alchemy looks awesome.  Not exactly flush with cash right now, but something to save for maybe...

When working with a single sound (like Iris does) in Rapture, there is no need to create an sfz file. Samples can be loaded directly into the elements. When working with multiple samples, if notepad is too much of a chore, there is at least one dedicated editor http://audio.clockbeat.com/sfZed.html 

In terms of CPU efficiency, it's pretty hard to beat Kontakt. I used it for many years on a low-spec machine and CPU was rarely a bottleneck. Mostly, it's just a RAM-eater.
 
Alchemy also does well in the CPU department, and it's closer to what you describe: a synth that uses samples, as opposed to a straight-up sample player. But it can get spendy when you start adding libraries. 
 
Omnisphere has to be mentioned, too. It is also a synthesizer that uses samples. It's surprisingly CPU-friendly. It comes with an enormous library of sounds. Only downside is it's pricey.
 
You might also want to look at more conventional synthesizers that are also able to load samples. Synthmaster is one. The upcoming Zebra3 will purportedly be able to do it (that one's definitely on my list for 2015). I understand that Image Line's Harmor can do it, but have no first-hand experience with it. I'm sure there are others.

Personally, I had high hopes for and tried Iris2 but they blew it in the timestretching realm...Who ever does this effectively first will get my attention...
 
Alchemy is an amazing and beautiful instrument....periodic fantastic sales... try it for free... and the player is free forever but you really want alchemy... I spent the evening with the player and immediately dug deep enough...basic library is huge and fantastic and often 50 percent off additional library sales...We're all awaiting A3 and it will likely take on the likes of Serum (which I like) and Iris... Camel hasn't rushed it and it is exciting to think of their next evolution... a great company!
 
Omnisphere also wonderful but never on sale...Skippy at PluginGuru has kept it alive with oooodles of fresh unique patches... 
 
Rapture is an amazing synth still...
 
U-he's Zebra2 allows for sample import also and is an amazing creation, although I've never explored this.... Urs is still quiet on the evolution of Zebra 3 but I'm hoping it will import waveforms/samples in a way that makes the entire process easy...and blow doors off the buildings; they've been coming out with so much stuff and yet continue working on Z3 so we're all waiting to see what Urs's new baby monster becomes!
 
I expect either U-he or Camel to make my own dream come true, which is to enable easy import of my vocal bits and creatively using them without the chipmonk effect...

Minor correction: Zebra2 only allows single-cycle waveform imports, and does not make it easy (a third-party tool is required). Z3 will have a wave import feature to make this more convenient, but won't actually have a proper sampler mode. Urs has said that "sampling isn't our cup of tea".
 
Alchemy would seem to be the best answer to the OP's original question. I've not gone further than fiddling with the demo (you can demo it without restrictions for a month!), but Camel claims it can do things to samples that no other synthesizer can pull off. The morphing and resynthesis features are certainly unique. I'd think it could handle your objective of playing back altered versions of your own voice.
 
 

Thanks again, bitflipper, Vastman, wealth of information here, as always.  Have been a Camel fan for years but never thought to look into Alchemy (even though I've gotten their emails about it every few months since it was realeased, haha).  Alchemy seems to be the answer to this question, so I'll call it resolved.  Sample manipulation and incorporation remains one of the most exciting aspects of recording for me, I don't necessarily need the banks but they really can save time (or kill time, depending).  Here's a song of mine, Iris featured, that demonstrates that sample magic:
https://smaltmine.bandcamp.com/album/the-dead
Thanks again, all!

That's some very original stuff, Jonathan. It always helps to hear someone's style when recommending tools. I think Alchemy's going to fit well into your world.

Thanks for listening and for the kind words bitflipper.