1) Chinese is impossibly difficult
The most widespread myth concerning Chinese is that it is practically impossible for non-Chinese to learn. This myth is commonly held both by outsiders and by the Chinese themselves, who will sometimes tell you that their language is the "most difficult in the world" with a kind of odd pride that their forebears managed to come up with such an impenetrable tongue. The truth is that while Chinese is certainly one of the hardest languages to master (at least for non-Asians), it is by no means impossible to do so.
The list of Westerners who have acquired fluent Chinese is by now quite long. You can see some examples in this previous post. Even in China people are starting to get less impressed when they hear foreigners speak their language well, although it can still attract amazement. The fact is that if you have a keen intelligence and a few years to dedicate to it, learning to speak Chinese to a functional level is perfectly possible.
Modern technology has also made things a lot easier. While remembering how to write Chinese characters by hand is indeed almost impossible unless you do nothing else but write Chinese for years and years (like Chinese schoolchildren), it is now almost never necessary to do so. Writing Chinese with a computer or a mobile phone is a lot simpler, as it only involves imputing the word in pinyin, and at most recognizing the character (which is many times easier than remembering how to write it). Electronic dictionaries are also incomparably easier to use than paper dictionaries, in which it is virtually impossible to find a character unless you already know how it is pronounced.
2) Chinese isn't really that hard
An opposite myth, peddled by Chinese language schools and sinologists and others who want to sell the language, is that Chinese isn't really as difficult as all that, especially if you use the right method. The reality is that mastering Chinese takes an awful lot of time and dedication no matter how it's done. Stories you hear about people who could speak fluent Chinese after only living in China for a year or so are to be treated with great skepticism.
You may hear claims that Chinese has an easy grammar, and this is more or less true. The point though is that the hard thing about Chinese is not its grammar. Chinese essentially has two aspects which make it difficult: the writing system and the tones. Learning the thousands of characters necessary to read the language is not unachievable for an adult, but it will take years of memorizing and practice, and there are no shortcuts. Claims that you "only" have to learn 3 or 4000 characters, as opposed to the dozens of thousands of words you need for English, ignore the fact that those characters can be put together in all sorts of ways to make up other words.
The tonal system is in some ways an even bigger obstacle than the writing: for those who did not grow up speaking a tonal language (most of humanity), it is just extremely unnatural to use tones to convey meaning. Learning to reproduce the four tones of Mandarin Chinese in isolation is easy enough, but learning how to use them smoothly while speaking is a huge feat. Even if you manage to memorize the tone of every syllable, actually convincing yourself to say the words like that is hard. Some people seem to have a natural ability to just pick up the tones by osmosis, but they are few and far between. Most people will just end up with toneless Chinese unless they make a conscious effort to learn the tones.
Add to all this the numerous characters which can have two or more different pronunciations according to their meaning, the hundreds of "four-characters proverbs" constantly used in writing, the high number of homophones guaranteed to confuse learners, the occasional use of traditional characters even in Mainland China etc etc... and you get the picture. Learning Chinese, while not impossible, is very tough. Just like learning Japanese or Arabic, it will take you years of work.
3) Learning the tones isn't necessary
This is a piece of advice which foreign learners sometimes get from well-meaning but misguided Chinese friends: don't worry about the tones, they aren't necessary. Even if you don't use them, "we Chinese will understand you anyway". There is a degree of truth in this: as long as you talk about pretty basic stuff in an otherwise correct fashion, most Chinese will be able to understand what you mean from context in spite of your lack of tones. However this changes as soon as you want to discuss deeper topics, or if you say a word in isolation (for instance, when you give a taxi driver the name of a place you want to go). The ability to understand toneless Chinese seems to vary from one person to another as well.
But there is an even better reason not to take this advice: if you speak Chinese without tones, it sounds pretty dreadful. People may understand you, but you are going to end up sounding extremely foreign for the rest of your life. It's a bit like speaking a European language and not conjugating any of the verbs. People can probably understand you anyway, but it's not a good reason to do so.
4) In many parts of China people don't speak Mandarin
You may hear people claim that Mandarin (here meaning 普通话 , China's official language) is only spoken in Beijing and will be almost useless in other parts of the country. This is nonsense. This state of affairs did hold true 100 years ago, and perhaps even 50. It is however quite untrue nowadays. The reality of the situation is that Mandarin is spoken just fine by all the young and the educated throughout the PRC (the exception is Hong Kong, because of its separate history and system). In fact, some of the other "dialects" (which might just as well be called languages) are unfortunately dying out in urban areas. In big cities you may struggle to hear anyone speak the local dialect.
Things do change when you go to the countryside, but in my experience even in little villages the young are able to communicate in good Mandarin, even if the middle aged and the elderly are not. As long as they don't go to Tibet or to remote areas, the chance that the average foreign traveller will meet anyone who can't communicate in Mandarin is tiny. Even when travelling in Guangdong, supposedly the province most attached to its own dialect, I had no trouble getting around with Mandarin.
It is true that people in the South of China sometimes speak Mandarin with a strange accent, and get some of the sounds wrong (in some areas L and N become mixed up, while in other areas it's H and F), but if your Chinese is up to scratch, it shouldn't be hard to understand people. Sometimes Mandarin spoken with a regional accent is enough to throw a foreign learner with basic Chinese, but that says more about their own abilities than it does about China's linguistic unity.
5) You can learn to speak Chinese without learning to read and write it
Due to the nature of the Chinese language, with its non-phonetic writing system, it would in theory be possible to learn to speak it without learning how to read and write it at all. You could learn how to say all the words, without ever learning the corresponding characters. The reality however is that it is extremely unusual for people to learn to speak Chinese without learning the writing as well.
Although there are certainly plenty of foreigners in China who pick up some basic Chinese phrases without knowing how to read at all, it is very hard to progress beyond that level without learning the characters at the same time. Any language course will teach you the writing system, and very few people have managed to get far with Chinese without learning it out of a book.
All the same, it is a widespread misconception among the Chinese that foreigners who speak their language are still likely to be quite unable to read and write it. If they think about it at all, they may assume that foreigners learn Chinese entirely through the medium of pinyin, or just by speaking it. They may also assume that Chinese writing is simply too difficult for foreigners to master. The following scenario has happened to me a number of times: I chat with a Chinese person in their language for a while, without them even commenting. Then at some point they see me read a message in Chinese on my mobile, give me a puzzled look, and say: "wow, you can actually read Chinese". Well d'oh.