I personally put stock in the many-worlds interpretation of quantum theory, and also believe that there is no "motion" of or through time, but that time is just a perceptual trait of any intelligent life observing the webbed structure of our space-time uinverse.
In this framework, I believe that there are no physical principles which exclude the possibility of backwards in time travel, nor would there be any paradox generated from such travel (including killing your grandpa... no problems there).
In reality, I think that the amounts and type of energy required for any significant travel of scale would be impractical if not impossible to acquire and apply at the specified location.
I predict however that we will someday acquire the technical means by which we can achieve small scale backwards in time travel, i.e. we can send an electromagnetic signal or perhaps even a particle based signal through time. I think that any such achievement will be based on artifically created time travel devices, and not naturally occuring physical phenomena, so that the range of possible backwards in travel time is limited to the lifespan of the time travel device.
This is also an explanation as to why we haven't received any signals (or visitors, in that unlikely case) from the future. Without the discovery and utilization of a naturally occurring "time gate," such signals and/or visitors could only visit times during which the time gate has existed. Naturally, if we haven't invented the device yet, then we are not yet in a time during which we can recieve such signals. I predict that as soon as the first time gate is constructed in a lab, it will receive signals from the future, even before the scientist use it for the first time to try and send such signals.