# locked out after reboot



## julian (Oct 27, 2010)

I've just rebooted and I'm now restricted to (video) console access of my server. Presuming it was a firewall problem I've disabled it - I've turned off  IPFW flushed/stopped and  edited them out of the rc.conf and still have no http/ssh access. 

What can I check to find out why? netstat shows that the services are listening on the correct ports (sorry it's a video console so I can't cut and paste.


----------



## SirDice (Oct 27, 2010)

What do you mean with "no access"? The services don't respond at all? Or unable to login?


----------



## julian (Oct 27, 2010)

I cannot login via ssh, telnet on port 25 gives Network unreachable,  and http will not load.


----------



## julian (Oct 27, 2010)

Sorry, I realise that I might not have answered question -- services do not respond.


----------



## SirDice (Oct 27, 2010)

Is there a firewall or other network equipment between you and the server that might be blocking things?

Since you have console access, try running tcpdump(1) to see if there's actually something arriving.


----------



## julian (Oct 27, 2010)

I seem to have inadvertently missed a post there:

http, smtp and ssh do not respond since reboot, therefore I am effectively locked out of my server. I have access through a video console and I have disabled the IPFW firewall  but still no response. I checked the ifconfig and that is fine and running so I am being blocked by something (something I presume I've done) but I do not know what. The services are listening on the correct ports. What can I run from the video console to see what is preventing the services from running normally.


----------



## SirDice (Oct 27, 2010)

julian said:
			
		

> What can I run from the video console to see what is preventing the services from running normally.


For starters run tcpdump(1). That will show you if network packets from your client actually arrive at the server. If there are no packets arriving it's not something on your server that doesn't work, it's the network or the client.

If there are packets arriving see if there's a proper three-way handshake on TCP connections.


----------



## anomie (Oct 27, 2010)

Depends on how you've "disabled" ipfw. Its default rule is to deny, so if you unload all other rules, guess what - you're denied access.  

It's been awhile since I have used ipfw, so hopefully this is the correct advice for you, but try: 
`# sysctl net.inet.ip.fw.enable=0`

This is documented in ipfw(8), BTW. If you're still not able to access services on your host, then next run tcpdump(1) like so: 
`# tcpdump -i interface_here port 22`

And try to ssh in. Are you seeing the beginning of a tcp handshake? (That should appear whether you have filtering enabled or not.)


----------



## julian (Oct 27, 2010)

Thanks for your help,


```
# tcpdump: no suitable device found
```
  (though i'm root) 
if I do: 

```
# tcpdump -i em0
tcpdump:listening on em0, link-type EN10MB (Ethernet), capture size 96 bytes
0 packets captured
0 packets received by filter
0 pockets dropped by kernel
```

Not sure if that is helpful.


----------



## SirDice (Oct 27, 2010)

Filter on SSH traffic for instance:
`# tcpdump -i em0 port 22`

Leave that running, then try to connect to your server from a client using ssh. You should see some packets arriving. If there's nothing arriving it's not your server that's the problem.


----------



## julian (Oct 27, 2010)

I get nothing from that: `tcpdump -i em0 port 30050` (custom port) - nothing at all what does that mean?


----------



## SirDice (Oct 27, 2010)

julian said:
			
		

> I get nothing from that: tcpdump -i em0 port 30050 (custom port) - nothing at all what does that mean?



It means the packets get dropped somewhere between your client and the server, i.e. the network.

NB that tcpdump will only show packets that are going to/from port 30050. The client must access that port or else you won't see a thing.


----------



## julian (Oct 27, 2010)

I tried to logon to the port but no packets, is the interface I'm using correct? I got it from ifconfig - would that be correct?


----------



## SirDice (Oct 27, 2010)

julian said:
			
		

> I tried to logon to the port but no packets, is the interface I'm using correct? I got it from ifconfig - would that be correct?



I don't know, you're the one that's looking at it, not me 

See if it has more interfaces, obviously pick the one where your clients connect to.


----------



## julian (Oct 27, 2010)

mmm I'm not really very practiced at this network stuff ;-) 

So I have em0 and em1 both active, I presume they are what connects the box to the network, and the lo0 which is the local loop - is that correct? (bear with me!)  So if I'm getting no tcp packets in or out (I can't send mail either from the command line or at least it doesn't arrive at destination), what could I have done to cause that to happen on a reboot or is it not my fault?


----------



## anomie (Oct 27, 2010)

Check which interface the IP you're trying to access is bound to (either em0 or em1). That is the one you should be referring to in the tcpdump invocation. Until you're using the correct interface, and attempting a tcp handshake from the client side, this test isn't telling you/us anything.


----------



## SirDice (Oct 27, 2010)

Have a closer look at the ifconfig output. Look at the flags portion, there should be an UP in there. If there isn't the interface is administratively down (ifconfig em0 down). A simple `# ifconfig em0 up` should bring it back up.

Also verify media and status. Status must be active, that means the interface detects a network signal. No signal means the cable might be broken or the switchport it's connected to is down. Media should indicate the proper speed/duplex settings for your network.



> mmm I'm not really very practiced at this network stuff ;-)


Understandable but as you are now finding out it really helps if you know how TCP/IP works


----------



## julian (Oct 27, 2010)

okay so I upped both em0 and em1 and they both now have UP in their flags.. progress ... so now I get packets received by filter:

0 packets captured
72 packets received by filter
0 packets dropped by kernel

still no connection though (though I'm thinking this is progress)

What can I test now?


----------



## anomie (Oct 27, 2010)

Silly question: are you able to take a photo of the output and upload it here? (It would be slightly easier to look at your dump session rather than try to explain what SYN and SYN-ACK packets look like.) 

Short answer to your issue: if you're seeing SYN packets from the client, but no SYN-ACK packets being returned by the host (and your service is actually listening on the port you think it is), then you still haven't disabled IPFW.


----------



## julian (Oct 27, 2010)

yes that's probably a good idea -- the window is small and can't be scrolled so if you would like me to output something please say :


----------



## anomie (Oct 27, 2010)

Now we're getting somewhere.  

See that warning about "no IPv4 address assigned"? That's a problem. Could you post a photo of your /etc/rc.conf contents? 

My guess is you (or someone) manually configured IP settings on that host, but didn't set things up so that they would survive a reboot. You're going to need something like the following in /etc/rc.conf: 

```
defaultrouter="10.0.101.1"
ifconfig_em0="inet 10.0.101.5 netmask 255.255.255.0"
```

Of course that's just an example, and you would need to use appropriate IP settings for your own network.


----------



## DutchDaemon (Oct 27, 2010)

And if you do more tcpdumps, please don't use a port, so we see at least some output (especially SYN/ACK packets). Use proto TCP if you must.


----------



## julian (Oct 28, 2010)

*aah joy!*



			
				anomie said:
			
		

> Now we're getting somewhere.
> 
> See that warning about "no IPv4 address assigned"? That's a problem. Could you post a photo of your /etc/rc.conf contents?
> 
> ...



Okay thanks everyone for your help, it was indeed a typical(?) beginner mistake, when I added something -- perhaps exim -- to the rc.conf I tried to be smart and used echo but being unfamiliar with the command used only one >  so I over wrote the file ...dohhh! Realizing my mistake I then put back everything I was running ie. apache, ssh etc but didn't put (didn't know) about the router configuration -- well that's what learning from your mistakes is all about (in public) :r

Thanks so much and see you when I make my next booboo (I hope):beergrin


----------



## DutchDaemon (Oct 28, 2010)

Thanks to julian for my 1,000th 
	

	
	
		
		

		
		
	


	




.


----------

