# Upgrading/Installing when I cannot login



## stream (Jun 10, 2021)

Hi,

I have 12+ running on my servers remotely, and wish to upgrade to 13.
I have a peculiar problem though. My system has been compromised by hacks who logged in as root via some backdoor unsecured interfaces to the server.
Now I cannot login as myself, it says /etc/login.conf is not owned as root.

I am able to  do a single user login as root. I cannot change /etc/ it says read-only. I tried various things such as chflags noschg etc.. doesnt seem to work.
Also,when I try freebsd-fetch, it says 
/var/db/freebsd-update not available.

So the standard methods don't seem to be working unfortunately. 

 Can you please help. How can I cleanup and upgrade my system


----------



## Alain De Vos (Jun 10, 2021)

When you boot in single user the root partition gets mounted as read-only.
Because people use this to perform fsck filesystem-check.
You can remount / in read-write using "mount -o rw /".
Then you will be able to edit any directory.


----------



## zirias@ (Jun 10, 2021)

stream said:


> My system has been compromised […]


Burn it with fire. Anything else is grossly negligent.


----------



## zirias@ (Jun 10, 2021)

To elaborate on that: I don't necessarily mean the hardware 

But a system that was once compromised can *never* be trusted again, there are just too many ways to hide malicious code etc…

If you can and need, try to save important data. But then, erase the disk and do a fresh install.


----------



## stream (Jun 10, 2021)

Thanks Alain, Zirias for the very quick responses. 

Zirias- when you say fresh install- you mean from disk correct?
Which requires someone physically going to the site.

Any other alternative to do a fresh install remotely?


----------



## covacat (Jun 10, 2021)

if you can access single user mode remotly you don't need to go there


----------



## Alain De Vos (Jun 10, 2021)

In single user mode you can tar cvfz and create a file of the directories you want to backup.
E.g. /boot , /usr/home , /etc/ , /usr/local/etc


----------



## stream (Jun 10, 2021)

Yes, I have created backup of all these dir previously. Also have snapshots.

What is the best way to re-install and/or upgrade from single user mode.
Assuming that I get past the /var/db/freebsd-update problem, which hopefully can be solved by mountting again.


----------



## zirias@ (Jun 10, 2021)

If this machine is housed at a hoster company, how did you install it initially?


----------



## stream (Jun 10, 2021)

Zirias- I did intial installs and upgrades all physically.


----------



## zirias@ (Jun 10, 2021)

Ok then I guess the only sane way out of that is: drive there. I know that's a bummer, but again, I would _never_ trust a system that was compromised at some point in time


----------



## _martin (Jun 10, 2021)

stream Take Zirias's advice to heart: you never trust a compromised server. Restore to a known state or do a fresh install. And most importantly -- figure out the way they got in and secure that. Because if you don't they will get in again.

As you have more servers I'd keep one offline for analysis.


----------



## grahamperrin@ (Jun 11, 2021)

Alain De Vos said:


> When you boot in single user …
> 
> You can remount / in read-write using "mount -o rw /". …



… or `mount -uw /` (I can't recall what led me to this habit). 

Another useful command for single user mode: 

`service zfs start`

Any number of other services can be started in the same way (or with _onestart_) although honestly, with a non-compromised machine I sometimes find it easier to exit to multi-user mode, than to figure out the range of things that should/must be started before an operation is performed in single user mode.


----------



## stream (Jun 11, 2021)

I can now login as myself. Would like to install fresh 13 version.
The last ditch is to install via thumb drive on-site.
Are there any other means to install the OS via internet?


----------



## Alain De Vos (Jun 11, 2021)

```
freebsd-update -r 13.0-RELEASE upgrade
```
And install afterwards.








						Chapter 25. Updating and Upgrading FreeBSD
					

Information about how to keep a FreeBSD system up-to-date with freebsd-update or Git, how to rebuild and reinstall the entire base system, etc




					docs.freebsd.org


----------



## stream (Jun 11, 2021)

I tried that earlier. It failed- says No mirrors remaining, and that 12.1 is unsupported for freebsd-update.


----------



## Alain De Vos (Jun 11, 2021)

SirDice on this forum might know something.


----------



## covacat (Jun 11, 2021)

i upgraded from 10.3 to  13 2 days ago
i needed to update to latest patch level of 10.3 before it worked


----------



## Alain De Vos (Jun 11, 2021)

So it works incremental. You must first go to the latest version of your current release before you can go to next release.


----------



## covacat (Jun 11, 2021)

this was the error i got and the solution was to update to max of the same minor version
The update metadata is correctly signed, but failed an integrity check.​


----------



## zirias@ (Jun 11, 2021)

Sorry for repeating myself, but: If you don't "wipe" the system, there's a risk. It was once compromised and there's no way to be sure what is left. Maybe, just maybe, you can do that from remote…


----------



## stream (Jun 11, 2021)

Zirias- yes, I do want to wipe. but first I would like to try a few hacks myself.

I see there is a problem with my internet connection. I can ping sites, but cannot access them via browser.
my rc.conf has defaultrouter="...." and gateway_enable="YES".

Does the order matter?


----------



## covacat (Jun 11, 2021)

stream said:


> Zirias- yes, I do want to wipe. but first I would like to try a few hacks myself.
> 
> I see there is a problem with my internet connection. I can ping sites, but cannot access them via browser.
> my rc.conf has defaultrouter="...." and gateway_enable="YES".
> ...


it does not
you may have a broken proxy  or some wrong fw rules


----------



## stream (Jun 11, 2021)

fw is all off.  I have no proxy


----------



## covacat (Jun 11, 2021)

does this work ?
fetch -o - http://www.freebsd.org/robots.txt


----------



## SirDice (Jun 11, 2021)

stream said:


> I see there is a problem with my internet connection. I can ping sites, but cannot access them via browser.
> my rc.conf has defaultrouter="...." and gateway_enable="YES".


DNS settings in /etc/resolv.conf? `gateway_enable` should only be turned on if you're routing traffic between interfaces. 



stream said:


> Does the order matter?


No, order is irrelevant.


----------



## stream (Jun 11, 2021)

covacat- thanks for the earlier message and tip regarding proxy.
I edited the dns servers and it is working fine now. 
upgrading freebsd.. lets see if it all burns and crashes


----------



## stream (Jun 11, 2021)

I just missed SirDice earlier message while replying to @covcat
Yes Sir I fixed the resolv.conf.

SirDice do you know if there is a way to wipe out the drive clean while upgrading? Is the only option a fresh install of the OS?


----------



## SirDice (Jun 11, 2021)

stream said:


> @SirDice  do you know if there is a way to wipe out the drive clean while upgrading?


That would be similar to trying to pull the rug from under the table. 



stream said:


> Is the only option a fresh install of the OS?


Yes, I'm afraid so.


----------



## stream (Jun 11, 2021)

SirDice said:


> That would be similar to trying to pull the rug from under the table.
> 
> 
> Yes, I'm afraid so.


haha-- exactly yes-- it should be possible with some magic and bit of luck.. to pull the rug from under the table.


----------



## SirDice (Jun 11, 2021)

Sure, not entirely impossible to do but quite tricky and error-prone.


----------



## stream (Jun 12, 2021)

SirDice said:


> DNS settings in /etc/resolv.conf? `gateway_enable` should only be turned on if you're routing traffic between interfaces.
> 
> 
> No, order is irrelevant.


Sir- can you examples of when gateway should be enabled. I don't remember when/why I had this setting. Probably when setting up  vmbhyve.


----------



## Alain De Vos (Jun 12, 2021)

Just put in that file

```
nameserver 8.8.8.8
```


----------



## stream (Jun 12, 2021)

Alain De Vos said:


> Just put in that file
> 
> ```
> nameserver 8.8.8.8
> ```


Alain- this is fine.. sorry my question was related to SirDice' comment that gateway should be enabled when routing traffic between interfaces.


----------



## SirDice (Jun 12, 2021)

stream said:


> can you examples of when gateway should be enabled. I don't remember when/why I had this setting.


You need `gateway_enable` to turn on IP forwarding. This means the system can route packets from one interface (your internal LAN interface for example) to another (external WAN interface for example). Or route traffic between two (or more) networks on various vlan(4) interfaces. Or any other ethernet interfaces (em(4),re(4), tap(4), tun(4), etc.)


----------



## Alain De Vos (Jun 12, 2021)

I don't know if this is taken care of by rc.conf. But in /etc/sysctl.conf.

```
net.inet.ip.forwarding=1
net.inet6.ip6.forwarding=1
```


----------



## zirias@ (Jun 12, 2021)

Alain De Vos said:


> I don't know if this is taken care of by rc.conf. But in /etc/sysctl.conf.


rc.conf(5):

```
gateway_enable
                 (bool) If set to “YES”, configure host to act as an IP
                 router, e.g. to forward packets between interfaces.

     ipv6_gateway_enable
                 (bool) The IPv6 equivalent of gateway_enable.
```
It does just that. As it's the documented way to enable routing, it's preferred to setting sysctl(8) values manually.


----------



## stream (Jun 12, 2021)

SirDice said:


> You need `gateway_enable` to turn on IP forwarding. This means the system can route packets from one interface (your internal LAN interface for example) to another (external WAN interface for example). Or route traffic between two (or more) networks on various vlan(4) interfaces. Or any other ethernet interfaces (em(4),re(4), tap(4), tun(4), etc.)


Thanks. I sort of get the general idea of ip forwarding in the context of routers. On a practical level though, If the host is running a VM, does the gateway_enable need to be enabled to get the VM to connect to the internet?


----------



## Alain De Vos (Jun 12, 2021)

With forwarding it will always work. Otherwise it might be dependent on the kind of network-virtualisation that is done.


----------



## SirDice (Jun 12, 2021)

stream said:


> I sort of get the general idea of ip forwarding in the context of routers.


Basically `gateway_enable` turns your system into a router. That's the simplest explanation. 



stream said:


> If the host is running a VM, does the gateway_enable need to be enabled to get the VM to connect to the internet?


Like Alain De Vos says, it depends on how you've set up the networking of the VMs or jails.


----------



## stream (Jun 14, 2021)

SirDice said:


> Basically `gateway_enable` turns your system into a router. That's the simplest explanation.
> 
> 
> Like Alain De Vos says, it depends on how you've set up the networking of the VMs or jails.


Does this pose a security risk in any way? i.e enabling the gateway.


----------

