# ssh session fails



## rtwingfield (Feb 22, 2016)

My ssh session(s) quickly fail after initial session is established, often within seconds, but sometimes the session will maintain "_carrier_" for hours . . .overnight into the following day.  Today, it has been virtually impossible to maintain a productive session, yet I have a running `sftp` session that does not fail, and I have maintained connectivity through the router(s) . . .actually logged in to the remote router, and also maintaining an active CCTV security remote DVR session video feed.

This phenomena occurs whether I am connected to remote alpha or bravo host server platforms.  Is is a problem with one of the routers?  (rhetorical question) . . .remote router is a Cisco RV082 and local is Cisco RV016.  The local client is PuTTY v0.60 (running on Windoze-7); have used for years . . .no problems.  To my knowledge, all of the usual suspects regarding heart-beat keep-alive techniques are in play, yet . . .boom, I'm really getting tired of that broken spring "sproing"  sound that PuTTY emits when the connection is lost . . .but I digress, so . . .what is the problem?

Problem is . . .there are no logged diagnostics; nothing in /var/log/messages, /var/named/named.log, nothing in any of the router logs; is there a log that I don't know about?  If this were a security certificate problem, then why do some sessions persist for hours, even days?  . . .otherwise, why failing withing seconds of initial session establishment?  Often, I don't even have time to `su`.

If this were an omission of a "heart-beat" keep-alive scenario, then why can I walk away from the session and  come back the next morning (assuming I've slept) and all is still good?  I keep coming back to the suspicion that a hardware component is failing, but then again, remember that `sftp`, router login sessions, and security DVR video feed sessions are stable.  I'm really trying to quit using profanity (in my old age), but WTF is going on here?


----------



## jrm@ (Feb 23, 2016)

Apparently PuTTy has an event log.  Maybe there is something interesting in there?  Have you changed any of the log settings for sshd so that it doesn't log to /var/log/messages?

ADDED: The -d flag for sshd and the LogLevel keyword for /etc/ssh/sshd_config might be useful.


----------



## rtwingfield (Feb 23, 2016)

I've all but forgotten that putty logs things, too.  I usually just ignore that - I'll check there.  As for the sshd, I'll have to find the configuration for the same.   Thanks!


----------



## Juha Nurmela (Feb 23, 2016)

Does it matter which shell you are using ? Does `ssh host tr -u a-z A-Z`, no terminal involved, show the same behavior ?

Juha


----------



## SirDice (Feb 23, 2016)

At one of the companies I work for I have the same issue lately. For me this is caused by a bad internet connection. The connections appear to drop somewhere half way the company office and the datacenter where I need to work. Terribly annoying. But I can say sysutils/tmux has saved my bacon more than once.


----------



## rtwingfield (Feb 23, 2016)

Hello everyone, this is just a quick note to thank all of you for your suggestions.  I'm currently in a time constraint and devoting attention to another "life" . . .I'm a musician and have a 1 o'clock studio appointment . . .getting ready for that.

Later this afternoon, I'll try your suggestions.

Thanks so much!


----------



## kevingnet (Apr 20, 2016)

rtwingfield said:


> Hello everyone, this is just a quick note to thank all of you for your suggestions.  I'm currently in a time constraint and devoting attention to another "life" . . .I'm a musician and have a 1 o'clock studio appointment . . .getting ready for that.
> 
> Later this afternoon, I'll try your suggestions.
> 
> Thanks so much!



There's several things you may try:
1) Running the ssh session locally, this would isolate things a bit, to possibly narrow down
2) The idea of extra logging from above posts, it's great, you can also run the session with the switch -v for more verbose output
3) Check the computer activity
4) If possible, do you have another network card to test with, in case it may be hardware related

Does it happen if you ssh from another computer?


----------



## rtwingfield (Apr 21, 2016)

First, thanks for your interest in this problem, and for your suggestions.

I'm using the Putty app. but I have also tried with a command line session specifying -v, too, with the same failure scenario.  The problem is intermittent -- some days almost no problem, yet others almost impossible to depend on.  The remote server site is serviced by a Cisco RV082 dual WAN router (ISP's are Comcast cable, and Aristotle wireless).  Seems that the Aristotle service is more stable, but then again, it has slower bandwidth, and the session is also subject to failure.  Here at local office, we're utilizing a Cisco dual wan RV016, but only one WAN . . .AT&T U-verse (static IP's with excellent bandwidth > 75 down).

Yesterday, I connected utilizing Putty and specified:

SSH/port22 as usual
Nagle's algorithm enabled (TCP_NODELAY option)
Enabled TCP_keepalives (SO_KEEPALIVE option) with 10 seconds between keepalive null packets.
. . .and this morning, the session is still connected and active.  On the other hand, I have experienced an apparently dependable session connection suddenly disconnect after several hours of dependable connectivity . . .yet with no logged information.  Very frustrating.  In fact, I just tried two times to connect via the same session configuration and both failed within fifteen seconds (but the first from yesterday is still _alive and well_. . .???).  Also, I'll add that I have a SFTP over SSH client connected and still active since yesterday afternoon.  (I know . . .not prudent , but as a test.)

Running ssh sessions locally are dependable when connected to any of several local servers.

The computer network traffic activity is not heavy, and is fire-walled against DDOS, denial of excessive ssh login attempts, etc.

I don't think network cards are the problem.  There is a second off-line server at the remote site and when on-line, the same scenario exist.



> _Does it happen if you ssh from another computer?_


 If connecting to the remote server from my local office, yes, but then all workstations are routing through the same routers and network; however, if on-remote-site where the server is located, LAN wireless access is very stable.  Note that this is not access through the wireless WAN ISP; however, the wireless LAN connection is via the Cisco router.  The server is inside the LAN.

Question:  The follow problem seems to now exist with the rsa/dsa key scenario.  I'm going to resolve the "unable to open file" complaint and see what happens.


```
Unable to use key file "C:\user\RTW\.ssh\id_dsa.ppk" (unable to open file)
Using keyboard-interactive authentication.
Password for rtwingfield@bravo.xyz.net:
```


. . .could this contribute to the instability?

Again, thanks for your interest,


----------



## SirDice (Apr 21, 2016)

rtwingfield said:


> ```
> Unable to use key file "C:\user\RTW\.ssh\id_dsa.ppk" (unable to open file)
> Using keyboard-interactive authentication.
> Password for rtwingfield@bravo.xyz.net:
> ...


No, it's not relevant.


----------



## rtwingfield (Apr 21, 2016)

> . . .could this contribute to the instability?


 Actually I didn't think so (mostly another rhetorical question).

I'm beginning to wonder if AT&T's U-verse network is dropping the connection?  I'm looking for problems on my "local" link in the network.  Considering that I experience the scenario whether I'm connected via either ISP of the remote, I doubt that Comcast or Aristotle are the problem; therefore, something nearer my end of the network.

It was a struggle to upgrade from AT&T's old legacy ADSL service to U-verse.  I had to jump through a lot of hoops (telephone calls and shouting matches) to "persuade" them to open port 22 for email, to allow my DNS server to resolve queries downstream of their DNS, etc.  What could they be doing . . .?  But in fairness to AT&T, my other connection classes seem ok.


----------



## Juha Nurmela (Apr 21, 2016)

What kind of surroundings you physically have? We have one location close to a (at times) busy metal shop, lots of large scale welding. No computers or networks involved, luckily, but radio interference is a nuisance.

Juha


----------



## rtwingfield (Apr 21, 2016)

Juha Nurmela said:


> What kind of surroundings you physically have? We have one location close to a (at times) busy metal shop, lots of large scale welding. No computers or networks involved, luckily, but radio interference is a nuisance.
> 
> Juha



Interesting . . .the remote server site is a 1950's era retired USAF two story building, originally built and used as a photo-recon processing lab during the cold war.  (Rumor is that U-2 camera footage/frames were processed there)  Lots of green tile, secure rooms, floor drains, 470v 3-phase electrical panels with labels like PhotoMat#1, #2, etc.

I am the current AR Wing Civil Air Patrol Information Technology Officer.  It is there that the problem server is located.  Actually, the servers and routers are in the came "closet" room where the HF and VHF radios are located.   Antennas on the roof, but . . .hum?


A little more thought -- the wireless ISP antenna is actually mounted on one of the masts from which the HF antenna is swung.  Also, not experiencing any problems with other SFTP, wireless LAN or WAN, networked CCTV security server, etc.  RFI is probably not a problem for us.


----------



## Juha Nurmela (Apr 21, 2016)

That intermittency sounded so familiar... No problems during the slow times, then suddenly an order for a crane arrives.

You've left an idle ssh session running overnight, `tcpdump -iem0 port 22`'ing and `ktrace -p`'ing both ssh and sshd pids, on both ends ?

Juha


----------

