To some extent it is normal, there are not enough details to say exactly what
is going on in your case.
I did a search for you, using the same keywords as your subject title:Internet traffic when not doing anything
The first hit, explains a little, as to why, etc. It is not something that only occurs
with "network manager" and Debian. Actually in the link below it is the modem, that indicates activity,..but in a nutshell , there all ways is some activity. It does not matter if is "network manager" or the modem, what ever , they all ways will show some activity. http://lifehacker.com/5979481/why-does-my-modem-blink-even-when-im-not-using-the-internet
You can determine what the activity is more specifically if you want. There are tools available to do that.
Again, a search will get you all kinds of details on that.On a Debian system, how can I see what the network activity is
Maybe the best place to start:https://wiki.debian.org/SystemMonitoring
==== additional note====
Personally, I do NOT use any "automatic" connecting,..I only go on-line, connect
when I have a reason, for example to check this forum, or other servers, after I am finished with my on-line tasks, I completely disconnect from internet, and stay disconnected , to do my off-line tasks / work.
I never just leave the computer connected to internet and un-attended, even for a few minuets.
Of course , a server is different and it is online 24 hours, 7 days ,...continuously.