If you find yourself in the situation in the midst of a test, yes, you just give up and move on. As you've noted, it takes WAY too much time to find the careless error and that kills your score in the end.
BUT there are some things you can do while you are studying to minimize these types of errors. You have some bad habits (as we all do), and your task is to identify them and build better habits during study so that you are not making as many careless errors. (Though note that you'll always make some - we're human, after all. :)
Read this article and follow what it says, especially for the "careless errors" part (defined as errors where you really did know exactly what to do, but you made one of those slap-yourself-on-the-side-of-the-head mistakes).
http://www.manhattangmat.com/articles/error-log.cfmNote that this is going to take some time / work - it's just like an athlete or a musician learning how to minimize their own mistakes / errors. It takes a lot of work to bread old bad habits and build new better ones, but you can do it!