Capacitive touch sensors require capacitive "common earth", for your body and the device. In addition, the capacity from your to this earth and from the device to this earth should be relatively stable. This "earth" is not always (if at all, not sure) related to the ground line. When you use your phone, the "earth" is you. The same happens when you touch the device. Note that the "connection" can be capacitive, so it works not only with touching metal parts but also with touching "in the near".
As I wrote before, I am not an expert in this field. The theoretical list of possible reasons: extreme floating the device earth, relative to you (f.e. something inside the power supply); too much isolating or static accumulating environment (table and floor, synthetic and natural covers); interferences from something in the near (some powerful electromagnetic field generator); or (still, sorry) problems with concrete device, which can work better in "better" environment.
The environment you can start to check with a phone or tablet (with capacitive screen). Try to use it putting on the table/floor/chair, NOT keeping it in hands but staying as far as possible from the device. Can you notice the reaction is (much/little) worse then usual?
You can also try to work with X-Touch in "unusual" position, f.e. sitting near it on the table (not touching the floor). Or taking it on lags. Is there are any changes in responsiveness?
All that sounds like a black magic, but when pF range capacity is in game, any voodoo in the near can drastically change the result
PS I am sitting now in the middle of the biggest physic scientific center in Europe... probably if I just go tomorrow outside the building and loudly ask how to deal with that problem, some world class expert in the topic will give an answer. But there is also a big probability there will be more then one... they will be in disagreement... start shouting on each other and I will be accused in organizing all that mess