The first clue is the animal's demeanor and behavior. A service animal should be very well behaved and obedient. A dog who wanders around at the end of the leash aimlessly probably isn't a service animal.
If you still aren't certain, you can ask the animal's owner if it is a service dog. Businesses are permitted to ask whether an animal is a service animal, and what tasks the animal is trained to perform that the human handler cannot do for themselves. The business is not, however, permitted to ask for information about the specific nature of the person's disability or other invasive questions. If an animal is not trained to perform tasks to mitigate the handler's disability, then it isn't a service animal under the ADA.
Guide dogs usually wear a special leather harness that helps them to guide their owner. In addition to the harness, the owner will also use a leash for controlling and directing the dog. Most other types of service animal wear some sort of marking such as a vest or cape, or special gear, like a harness. However, not all service dogs will wear special markings. Under the ADA, they aren't required to be marked. Unfortunately, the presence or absence of a cape or gear alone doesn't make it clear whether or not an animal is really a service animal.
Unfortunately there are unscrupulous people who buy gear over the Internet to try to pass their pets off as service animals. Sadly, this is also true with certification. Anyone can forge their own certification or purchase fake certification over the Internet for their pets. Many states have begun taking steps to prosecute those who falsely claim their pets as service animals with stiff fines and jail time.