# Nvidia Adds Telemetry To Latest Drivers



## MarcoB (Nov 7, 2016)

As a Nvidia user I'm a little bit concerned about this development:
https://yro.slashdot.org/story/16/11/07/1427257/nvidia-adds-telemetry-to-latest-drivers


----------



## x-com (Nov 7, 2016)

Well, perhaps Linux and *BSD will have a little time before that happens. The article mostly talks about the Windows Version, but we all should have an eye on that. But that's a trend anyway when using MacOS or Windows, most components will send telemetry data, applications will follow, if they aren't doing so until now. Gaming portals like steam or origin are spying since they appeared. 
Even Ubuntu sends search data to Amazon, so we really should be on the watch and hope we can avoid this at least in the open source field.


----------



## SirDice (Nov 8, 2016)

x-com said:


> The article mostly talks about the Windows Version, but we all should have an eye on that.


One of the reasons the NVidia drivers work so well on Linux and BSD is mainly because they're identical. The only differences are the hooks to the kernels and the OS, but the actual driver itself is mostly the same. 

I'm not too worried about usage statistics or error reports but it would be nice if it was opt-in (heck, even opt-out would do) instead of being forced down everyone's throat.


----------



## x-com (Nov 8, 2016)

SirDice said:


> I'm not too worried about usage statistics or error reports but it would be nice if it was opt-in (heck, even opt-out would do) instead of being forced down everyone's throat.



Well, you should be worried, since every piece of information is linked to all other information about you and your systems. Under Windows NVidia forces you to either have a google or nvidia account if you want to use their auto-update tool thats a good hint what they plan to do: Big Data!
Windows, MacOS and Android are spying you out, and so do most of the applications available for them, now even the drivers (or at least their supplemental software) start doing this. This gives nice profiling data.
By the way: This is a potential security problem, what if any of those telemetry non-sense has a bug!
Linux and BSD's were an relatively safe alternative to avoid all this, but Canonical with its desktop search being linked to Amazon or Steam comming to Linux/BSD is no good sign. So keep an watchfull eye on this if you want to keep your data.


----------



## MarcoB (Nov 8, 2016)

I'm not worried if the information is just error reports, but the point is that you never know for sure what it sends to Nvidia (and of course the security issue). So I'm avoiding spyware as much as possible. The real "problem" here is that the driver is closed source, and Nvidia can put in whatever they like. It is after all their product.

Only way out would be an open source driver for Nvidia cards, or get another brand videocard. So does anyone know if the nv driver is ok on FreeBSD? And what is the best alternative for a graphics card?


----------



## x-com (Nov 8, 2016)

MarcoB said:


> Only way out would be an open source driver for Nvidia cards, or get another brand videocard. So does anyone know if the nv driver is ok on FreeBSD? And what is the best alternative for a graphics card?



Depends on your graphics card, here you can read the supported models and drivers, most of the newer cards need nvidia
https://wiki.freebsd.org/Graphics

Concerning the error informations, do you know what Windows 10 is sending, when enabling full crash reports? Even the documents you were working on! Very trustfull


----------



## MarcoB (Nov 8, 2016)

x-com said:


> Concerning the error informations, do you know what Windows 10 is sending, when enabling full crash reports? Even the documents you were working on! Very trustfull


I think this really is a bad development, but all corporations in Silicon Valley are into this these days, and it isn't going away. It all boils down to trust, but since all manufacturers are now getting into "big data" and telemetry, this trust is gone.

Imo only way to escape from it is to use oss all the way. The Nvidia binary blob has always been a bit of an achilles heel on BSD and Linux.


----------



## drhowarddrfine (Nov 8, 2016)

I just woke up so my thinking cap isn't on but how would a driver be able to collect personal information about you and your data?


----------



## x-com (Nov 8, 2016)

MarcoB said:


> Imo only way to escape from it is to use oss all the way. The Nvidia binary blob has always been a bit of an achilles heel on BSD and Linux.


That's right, but unfortunately even nouveau under linux is no real alternative at least when you need 3D support you have to use the binary driver.



drhowarddrfine said:


> I just woke up so my thinking cap isn't on but how would a driver be able to collect personal information about you and your data?


Most certainly not the driver itself, but I guess the original install script taken from NVIDIA will install an additional software for that. Under Windows this is located in the so called GForce Experience. At the moment this is not available for Linux/UNIX, that's why I said we should have an eye on that.


----------



## roddierod (Nov 8, 2016)

GeForce Experience is Nvidia online gaming venture, so if you don't use that all the telemetry stuff is probably irrelevant as the driver can't collect the personal, as drhowarddrfine alludes to.

I'm using GeForce Experience because it allows me to stream games from my PC to my Nvidia Shield.
The OP's link contains a link to a software that will easily allow you to disable the telemetry on Windows.

I would guess there would be valid reason for wanting telemetry data for online gaming (best servers to connect to, other online games in the same area...yadda)...I don't think there were nefarious reason behind Nvidia doing this. And giving what GeForce Experience is and does I don't think BSD will be worrying about this anytime soon.  The Shield does run AndroidOS so maybe it will get to Linux sooner, but I'm not sure how close AndroidOS is to any Linux now.


----------



## kpa (Nov 8, 2016)

If I was an engineer at NVidia the very first thing I would like to is know how well the drivers I'm working on perform and work on different pieces of NVidia hardware. The telemetry is exactly what I would insist on implementing on the drivers to collect the performance data over very large number of users.


----------



## x-com (Nov 8, 2016)

roddierod said:


> The OP's link contains a link to a software that will easily allow you to disable the telemetry on Windows.



That only lasts to the next update on Windows as Microsoft prooved during the last updates


----------



## x-com (Nov 8, 2016)

kpa said:


> If I was an engineer at NVidia the very first thing I would like to is know how well the drivers I'm working on perform and work on different pieces of NVidia hardware. The telemetry is exactly what I would insist on implementing on the drivers to collect the performance data over very large number of users.



Well as user I would respond that it's simply none of your business how I use the card, if you want that data go and perform your tests in your labs not on my pc. That's an old discussion but since advanced tools for data analysis exists and are used to interpret even the most rediculus data I consider even this "telemetry" data as a risk. When I see how many applications now gather telemetry, hell even Firefox will do if you don't opt out, this creates a massive amount of data which alltogether form a meaningful profile of your computer usage. That's what one should avoid at all costs.


----------



## roddierod (Nov 8, 2016)

x-com said:


> That only lasts to the next update on Windows as Microsoft prooved during the last updates



I never let Windows auto update, so that not much of an issue for me.


----------



## kpedersen (Nov 9, 2016)

Exactly, who the hell would just allow random updates to run at random times? For people who aren't mad, there exists http://download.wsusoffline.net/

Also, if anyone is worried about how Windows and software running on Windows is simply not suitable for the internet due to leaking data, just disable the network entirely. Then to access the internet, just run a disposable VM (since it is disposable, it might as well be running windows too) which does connect to the network.

Microsoft even provide disposable images for this reason: https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/

Think of it not as a Virtual Machine but as a heavily sandboxed web browser. Then think of your host Windows not as an OS, but as a proprietary compatibility layer bios extension.


----------



## roddierod (Nov 9, 2016)

Actually, until I started using the Geforce Experience to stream stuff to my TV I kept my network adapter disabled at all times in Windows.  I never use my Windows home machine for anything on the internet.

Now I need to just look into not allow my Windows install internet access...I'm sure my router can handle that.


----------



## x-com (Nov 9, 2016)

Just tell an average user how to set up a wsus. They got them by the balls and everybody is riding that train. These things need to be undone and users need to be more aware of this!


----------



## roddierod (Nov 9, 2016)

Average user don't care about this stuff. They just want to click click and it all magically happens and they can post every detail of their life on social media.  It is the about average user that argues about this stuff.


----------



## obsigna (Nov 10, 2016)

Regarding NVIDIA, I am not directly affected, however, NVIDIA's so called Telemetry is not a single case, but only another manifestation of the business culture 4.0 - turn your customers into commodity, while precautionarily don't ask them, because they may say "no".

Clearly the decision makers of said BC4, don't have mother and father, who could have told them "Do not open/read letters (messages, informations, ...) that are not intended to reach you!" My mother told us children, and even when her directive at that time was limited to letters only, of course, in the 21st century this still holds for any kind of successors of regular letters and other informations on paper/tape.

The drawback of the internet is, that everybody of us is faced to countless ill-bred individuals and their organizations, and the NVIDIA case raises once again the main question, how to stop being lumbered with all these useless loans from all this rude plebs.

Since the internet comes also with benefits, the radical answer, simply not to use it with your favorite device, but do everything out of your personal jail, is quite understandably not to everybody's liking.

I am more in favor of filtering DNS requests. One quite common approach is by the way of hosts(5) files, like http://pgl.yoyo.org/adservers/, http://someonewhocares.org/hosts/, http://winhelp2002.mvps.org/hosts.htm, http://www.hosts-file.net/. These may be deployed on a single machine or also on a gateway. Actually, I do this already on my FreeBSD Home Server and on my FreeBSD on AWS-EC2 VPN-gateway by feeding unbound.conf(5) with ~20000 void-zones, which were compiled form said hosts listings. I easily could add another void-zone like "tm.nvidia.com", however, I am still looking for more sophistication.

Does somebody know specialised hosts compilations of domains used by software/devices for calling home?

Are there any better suggestions for properly maintained hosts listings? I am still looking for listings of ad, tracking, and other malware domains in Brazil.

Are there any suggestions for more sophistication, or perhaps more effective approaches?


----------



## roddierod (Nov 10, 2016)

obsigna I totally agree with with your view on "business culture 4.0", this is the major cause of telemetry in applications.  I been preaching this for years about Facebook, Google Et al. but most people don't listen because these things are just too convenient.

I'm using http://winhelp2002.mvps.org/hosts.htm for my hosts file and manually adding things myself. I don't know of any specific sites with domains that call home, but I will post if I find one.


----------



## BSD-Kitsune (Nov 10, 2016)

Another way would be to collect network data via sockstat to figure out which port the telemetry is going out on, then block the ports in PF and your gateway router.


----------



## x-com (Nov 10, 2016)

Does anyone of you see any chance that these telemetry non-sense or "business culture 4.0" will vanish in the foreseable future or how you could the average user make more aware of it?


----------



## MarcoB (Nov 10, 2016)

obsigna said:


> I am more in favor of filtering DNS requests. One quite common approach is by the way of hosts(5) files, like http://pgl.yoyo.org/adservers/, http://someonewhocares.org/hosts/, http://winhelp2002.mvps.org/hosts.htm, http://www.hosts-file.net/. These may be deployed on a single machine or also on a gateway. Actually, I do this already on my FreeBSD Home Server and on my FreeBSD on AWS-EC2 VPN-gateway by feeding unbound.conf(5) with ~20000 void-zones, which were compiled form said hosts listings. I easily could add another void-zone like "tm.nvidia.com", however, I am still looking for more sophistication.



At the moment I'm using adblock to block all ads and trackers. Does the hosts file method has any advantages over that?


----------



## x-com (Nov 10, 2016)

MarcoB said:


> At the moment I'm using adblock to block all ads and trackers. Does the hosts file method has any advantages over that?



Well, just read the faq of the first link (yoyo): Addblock does only work with certain browsers. The host file method affects the whole operating system (in short terms)


----------



## roddierod (Nov 10, 2016)

x-com said:


> Does anyone of you see any chance that these telemetry non-sense or "business culture 4.0" will vanish in the foreseeable future or how you could the average user make more aware of it?



Personally, I definitely do not see the business culture changing anytime soon. Everything is about the data. There is too much money in it.  Even things that seemingly would not need data are using it. 

I just sat through a keynote at software conference and they gave an example about how a middle school principal used a database to figure out he should call a students parents and then was able to coordinate consulting for the child. Now, back in the day a school principal would not need to consult a database to figure this out...let alone be so proud of his inability to deal with children in a social setting that he endorses some software that helped him figure what he should do.

I don't think you can make average people care about something that could possibly cause them inconvenience.


----------



## x-com (Nov 10, 2016)

roddierod said:


> Personally, I definitely ...



Sad but true, I guess


----------



## T-Daemon (Nov 10, 2016)

obsigna said:


> .... http://pgl.yoyo.org/adservers/, http://someonewhocares.org/hosts/, http://winhelp2002.mvps.org/hosts.htm, http://www.hosts-file.net/. ...


I like to point out, who is interested in blocking harmful and undesired domains with the help of hosts files, there is, in my opinion, a very good source for that. It's a repository which extends and 
consolidates existing hosts files from other sites, and merges them into a unified hosts file with duplicates removed. There are hosts files ready to download to choose from by category (adware, malware, gambling, porn, social, or in combination). Even one can generate a own, tailored unified hosts file by adding extra sources.

Unified hosts files blocking unique domains from 29,749  to 35,992 at the time of this writing.

https://github.com/StevenBlack/hosts


----------



## MarcoB (Nov 10, 2016)

T-Daemon said:


> I like to point out, who is interested in blocking harmful and undesired domains with the help of hosts files, there is, in my opinion, a very good source for that. It's a repository which extends and
> consolidates existing hosts files from other sites, and merges them into a unified hosts file with duplicates removed. There are hosts files ready to download to choose from by category (adware, malware, gambling, porn, social, or in combination). Even one can generate a own, tailored unified hosts file by adding extra sources.
> 
> Unified hosts files blocking unique domains from 29,749  to 35,992 at the time of this writing.
> ...



This seems a good one. With the other ones I still get ads, so I'll try this one for a while. Also, FF uses around 120MB less memory without AdblockPlus on my system.


----------



## obsigna (Nov 10, 2016)

MarcoB said:


> At the moment I'm using adblock to block all ads and trackers. Does the hosts file method has any advantages over that?



I put the tools that I use for DNS filtering with unbound(8) on GitHub -- https://github.com/cyclaero/void-zones-tools. The README file discusses the pros & cons of the various methods and there is also a section on _"How does this compare to Browser Plugins?"_



> ...
> 
> Browser plugins are destined to one piece of software and not to the whole machine. Void zones are active for the whole machine or in the case of a gateway, for any number of clients, and even for those (Android) which don't allow ad-blocking plugins.
> 
> ...


----------



## obsigna (Nov 10, 2016)

T-Daemon said:


> I like to point out, who is interested in blocking harmful and undesired domains with the help of hosts files, there is, in my opinion, a very good source for that. It's a repository which extends and
> consolidates existing hosts files from other sites, and merges them into a unified hosts file with duplicates removed. There are hosts files ready to download to choose from by category (adware, malware, gambling, porn, social, or in combination). Even one can generate a own, tailored unified hosts file by adding extra sources.
> 
> Unified hosts files blocking unique domains from 29,749  to 35,992 at the time of this writing.
> ...



Nice compilation, I am trying the Window 10 telemetry list from that repository with my void-zones-tools for some days, and I will add it to my void-zones-update.sh script when this won't disturb the desired operation of my 2 Windows 10 clients.


----------



## MarcoB (Nov 10, 2016)

obsigna said:


> I put the tools that I use for DNS filtering with unbound(8) on GitHub -- https://github.com/cyclaero/void-zones-tools. The README file discusses the pros & cons of the various methods and there is also a section on _"How does this compare to Browser Plugins?"_


This looks really interesting. Why isn't this in ports? 
First I'll have to dive into unbound() for this.


----------



## obsigna (Nov 10, 2016)

MarcoB said:


> This looks really interesting. Why isn't this in ports?


The usual excuse for laziness. I thought nobody would like it.


MarcoB said:


> First I'll have to dive into unbound(1) for this.


Two years ago, I described the method including setting-up Unbound in two posts on my BLog. This is in German language, however, using an online translation tool, perhaps it might be still suitable to get you started:

http://blog.obsigna.net/?p=504
http://blog.obsigna.net/?p=509

PS: The tools presented in the 2 years old BLog posts still do work, however, I suggest to use the updated ones on GitHub.


----------



## MarcoB (Nov 10, 2016)

My german isn't bad (I'm from NL) so I'll read and try to understand the german text first.


----------



## obsigna (Nov 10, 2016)

MarcoB said:


> My german isn't bad (I'm from NL) so I'll read and try to understand the german text first.


In case of any doubts you are welcome to send me a private e-mail. You'll find the address on my BLog - Impressum.


----------



## roddierod (Nov 11, 2016)

obsigna said:


> The usual excuse for laziness. I thought nobody would like it.


I have been looking for something that would do what what you are doing here, namely the blocking all subdomains of a domain.

One question, say I wanted to block just the ads on a domain will just entering something like: 
0.0.0.0 ads.exampledomain.com

block just that but allow www.exampledomain.com, or is my understanding incorrect?


----------



## obsigna (Nov 11, 2016)

roddierod said:


> I have been looking for something that would do what what you are doing here, namely the blocking all subdomains of a domain.
> 
> One question, say I wanted to block just the ads on a domain will just entering something like:
> 0.0.0.0 ads.exampledomain.com
> ...



Your understanding is correct. Once the tool hosts2zones converted this to an empty (void) Unbound zone entry ...

```
local-zone: "ads.exampledomain.com" static
```
... Unbound would respond with NXDOMAIN for requests to ads.exampledomain.com itself and all of its subdomains, but, it would process requests to www.exampledomain.com normally, i.e. either forward or recursively resolve it.

Perhaps you already realized it, anyway it is good to emphasize it again. The tool hosts2zones does join subdomain host entries into one zone for the highest common level domain. By this way the number of void-zones are significantly reduced. For example, today's run of void-zones-update.sh turned 26849 hosts file entries into 17648 void zones.


----------



## MarcoB (Nov 11, 2016)

obsigna said:


> The usual excuse for laziness. I thought nobody would like it.
> 
> Two years ago, I described the method including setting-up Unbound in two posts on my BLog. This is in German language, however, using an online translation tool, perhaps it might be still suitable to get you started:
> 
> ...



Unbound is running and working. Trying to install a working /var/unbound/local-void.zones now. I've followed the directions on your site http://blog.obsigna.net/?p=509 but the binary hosts2zones coredumps with an "illegal instruction".


----------



## obsigna (Nov 11, 2016)

MarcoB said:


> Unbound is running and working. Trying to install a working /var/unbound/local-void.zones now. I've followed the directions on your site http://blog.obsigna.net/?p=509 but the binary hosts2zones coredumps with an "illegal instruction".


Did you use the software from 2014 from my BLog post or are your referring to the updated one on GitHub?

PS: just checked the tool from 2014 and it compiled and worked fine on my machine FreeBSD 11.0-RELEASE-p3, amd64, Intel(R) Atom(TM) CPU D510  @ 1.66GHz (1666.72-MHz K8-class CPU)


----------



## MarcoB (Nov 11, 2016)

Ah, sorry my fault. Got the wrong one from 2014. I'll try the one from github. But do I have to install github first? The command "git" doesn't work out of the box.


----------



## obsigna (Nov 11, 2016)

You can fetch the .zip-archive of the repository.
`fetch -o void-zones-tools-master.zip https://github.com/cyclaero/void-zones-tools/archive/master.zip`
`unzip void-zones-tools-master.zip`

However, I find it still strange that the "old" tool crashes on your machine. If the new one crashes as well, then please make the tool with the command `make clean install CDEFS="-march=native"`, and then try again.


----------



## MarcoB (Nov 11, 2016)

New tool crashes as well. The directory /usr/local/etc/void-zones is created and filled with the hosts files. However /var/unbound/local-void.zones is still empty and the binary coredumps with an illegal instruction.
...And same with `make clean install CDEFS="-march=native"`


----------



## obsigna (Nov 11, 2016)

MarcoB said:


> New tool crashes as well. The directory /usr/local/etc/void-zones is created and filled with the hosts files. However /var/unbound/local-void.zones is still empty and the binary coredumps with an illegal instruction.
> ...And same with `make clean install CDEFS="-march=native"`


Please can you post some details of your machine? Which FreeBSD version (32 or 64bit) and which CPU?
Please can you edit the Makefile, changing the CFLAGS -g0 to g and -Ofast to -O0. Then make install and run again. Then it would be great if you could upload the coredump to somewhere where I can  pick it up.


----------



## MarcoB (Nov 11, 2016)

Sure I have a dual Xeon 2.8 GHz (Nocona) with 4 GB RAM. Uname -a: FreeBSD yokozuna.lan 11.0-STABLE FreeBSD 11.0-STABLE #0 r307320: Fri Oct 14 21:19:48 CEST 2016     root@yokozuna.lan:/usr/obj/usr/src/sys/YOKOZUNA  amd64. I try the Makefile edit.


----------



## obsigna (Nov 11, 2016)

I received and analyzed the core dump. For some reasons, your Xeon does not like one or some SSE instructions. So, for the time being it will help to disable it.

Please edit the file binutils.h. On line 151 please replace #if defined(__x86_64__) with #if 0. By this way all functions using SSE will be disabled and the non-vector versions will be used instead. Build & install again, and now it should work.

`make clean install`

I need to investigate why the Xeon got a problem with some SSE instructions.


----------



## MarcoB (Nov 11, 2016)

Compiling and running went fine now. Got a new error though:

```
# service local_unbound restart
Stopping local_unbound.
Waiting for PIDS: 87910.
Starting local_unbound.
/var/unbound/local-void.zones:1: error: syntax error
read /var/unbound/unbound.conf failed: 1 errors in configuration file
[1478906271] unbound[87948:0] fatal error: Could not read config file: /var/unbound/unbound.conf
/etc/rc.d/local_unbound: WARNING: failed to start local_unbound
```


----------



## obsigna (Nov 11, 2016)

Please show your file /var/unbound/unbound.conf and please send me the generated file /var/unbound/local-void.zones by the same way like you send to me the core dump.


----------



## obsigna (Nov 11, 2016)

The generated file /var/unbound/local-void.zones was OK.

I experimented a little bit with your file unbound.conf, and I found out that any local-zone: directive must come before the forward-zone: directives that are included by the file /var/unbound/forward.conf. So please move the include directive of /var/unbound/local-void.zones before all the other includes. The following did not error out:


```
server:
        username: unbound
        directory: /var/unbound
        chroot: /var/unbound
        pidfile: /var/run/local_unbound.pid
        auto-trust-anchor-file: /var/unbound/root.key

include: /var/unbound/local-void.zones
include: /var/unbound/forward.conf
include: /var/unbound/lan-zones.conf
include: /var/unbound/control.conf
include: /var/unbound/conf.d/*.conf
```


----------



## MarcoB (Nov 12, 2016)

Yes, and we have liftoff! Thanks for all your help. FF is faster and uses a lot less memory so this is a really nice tool. Some ads are still coming through but that's a matter of updating the list.

Another thing I'll have to do is learn a bit about DNS, the only site so far that is unreachable after running unbound is my cloud storage in FF.


----------



## rigoletto@ (Nov 12, 2016)

Just in regards of browser add on, there is uBlock Origin what include everything AdBlock include and *a lot of more*.


----------



## obsigna (Nov 12, 2016)

lebarondemerde said:


> Just in regards of browser add on, there is uBlock Origin what include everything AdBlock include and *a lot of more*.


You cannot beat telemetry of any kind of software, let it be the driver package of NVIDIA, or the various calling home daemons in Windows 10, or of software on other devices in your home network (Smart TVs, Playstations, Printers, ..., zillions of IoT devices ante porta) with just another browser plugin -- not with the very best of the world.

I am sure that filtering at the DNS (with a Hosts file or by the way of void zones) + perhaps some blocking at the Firewall is much more promising for dealing with the issue which was risen in the present thread.


----------



## rigoletto@ (Nov 12, 2016)

Hi obsigna ,

I was not pointing it out for this specific purpose, I was just willing to add an alternative to AdBlock in general (a lot of better IMO), since there is talk about AdBlock from people using that.

Also, they compile several lists beyond ads, including malware etc. and may be a good centralized source for them.

EDIT:

And yes, I totally agree that firewall/dns should be the most effective way to beat telemetry, proxy maybe a good too but I bit more complicated to implement per average user. But the most important is to have a source of what shall be blocked.

There are several lists, but one source where we can find everything would be a lot of better. I am not talking about uBlock, of course, that is just the best source I could find for now but it have other purposes than telemetry.


----------



## MarcoB (Nov 12, 2016)

I have to admit that an adblocker plugin does filter the ads in webpages better. But using unbound has some other advantages I really like e.g. not needing plugins anymore, FF needing a lot less memory, via unbound protects my complete system and I can switch to webbrowsers without adblockplugin. And of course it's just fun to fiddle with it .


----------



## obsigna (Nov 12, 2016)

MarcoB, I updated the GitHub repository of the void-zones-tools, and among other tiny details, I added a change that is supposed to fix the invalid instruction issue on your Xeon CPU. I cannot test it on my side, though. Please may I ask you to test it for me on your system, and let me know the result?

`fetch -o void-zones-tools-master.zip https://github.com/cyclaero/void-zones-tools/archive/master.zip`
`unzip void-zones-tools-master.zip`
`cd void-zones-tools-master`
`make install clean`
`void-zones-update.sh`


----------



## MarcoB (Nov 12, 2016)

The tools build and install ok. Running the script results in a new download of the txt-files, but then a coredump again with an illegal instruction.


----------



## obsigna (Nov 12, 2016)

MarcoB said:


> The tools build and install ok. Running the script results in a new download of the txt-files, but then a coredump again with an illegal instruction.



For the time being, please edit the file binutils.h, replacing once again on line 151 #if defined(__x86_64__) with #if 0, then build and install again. Obviously, this one needs more investigation.


----------



## MarcoB (Nov 12, 2016)

Ok I will. With 
	
	



```
#if 0
```
 it runs fine.
BTW do you know where unbound stores it's cache?


----------



## rigoletto@ (Nov 12, 2016)

IIRC Unbound store the cache in memory, but I think there is a way to make it store in a file.
You can use this guide to optimize Unbound if you like.


----------



## rigoletto@ (Nov 12, 2016)

obsigna 

I installed the void-zones-tools here and it is working very well, but there is a simple way to add more lists? I let the integrated Opera adblocker and the uBlock still on and some times they still get about 15 _things _in total on a single site.

Thank you!


----------



## obsigna (Nov 12, 2016)

MarcoB said:


> Ok I will. With
> 
> 
> 
> ...


Well, I got the next incarnation of binutils.h for testing, s. attached file. If you would be willing to assist in trouble shooting the Xeon issue, then please replace the original one by the attached binutils.h, and try again.

Also, it might be better to continue the troubleshooting session per e-mail, in order to reduce the noise on the forum, what do you think. If this is OK for you, then please return your results by e-mail.


----------



## obsigna (Nov 12, 2016)

lebarondemerde said:


> obsigna
> 
> I installed the void-zones-tools here and it is working very well, but there is a simple way to add more lists? I let the integrated Opera adblocker and the uBlock still on and some times they still get about 15 _things _in total on a single site.
> 
> Thank you!


The first run of void-zones-update.sh should have created the directory /usr/local/etc/void-zones/. In addition it should have placed a template my_void_hosts.txt, and this one is meant for adding additional domains.

Use 0.0.0.0 for blacklisting, and 1.1.1.1 for whitelisting. You may move this file to a more convenient location for frequent editing, and leave a symbolic link to it in /usr/local/etc/void-zones/.

For running the hosts2zones conversion without updating (downloading) all the remote Hosts files, you may want to use the following shell script:

```
#!/bin/sh

ZONES_DIR="/usr/local/etc/void-zones"
/usr/local/bin/hosts2zones /tmp/local-void.zones \
                           "$ZONES_DIR/my_void_hosts.txt" \
                           "$ZONES_DIR/pgl_void_hosts.txt" \
                           "$ZONES_DIR/sowc_void_hosts.txt" \
                           "$ZONES_DIR/mvps_void_hosts.txt" \
                           "$ZONES_DIR/mdl_void_hosts.txt" \
                           "$ZONES_DIR/away_void_hosts.txt" \
                           "$ZONES_DIR/ucky_void_host.txt" \
                           "$ZONES_DIR/telm_void_hosts.txt" \
  && /bin/mv /tmp/local-void.zones /var/unbound/local-void.zones
```
Anybody is welcome to post their custom my_void_hosts.txt as an issue on the GitHub repository of the void-zones-tools, and I will place another remote Hosts file for download on GitHub.


----------



## MarcoB (Nov 12, 2016)

Afaics only place where a lot of ads are not blocked is on Linkedin. These are mostly "sponsored content". Is blocking this kind of ads possible with the tool? Or do we really need a browser plugin for this?


----------



## rigoletto@ (Nov 12, 2016)

obsigna

I will try it. Anyway I want to add just the lists I find on uBlock Origin that are not already on void-zones-tools - _at least for now_.

The Opera integrated adblock use (IIRC) Easylist what is also on uBlock Origin, but the integrated adblock is faster than the plugin.

Thank you!


----------



## obsigna (Nov 12, 2016)

With the help of MarcoB, the Xeon invalid instruction issue with the hosts2zones tool has been resolved, and I updated the GitHub repository void-zones-tools already.


----------



## obsigna (Nov 12, 2016)

lebarondemerde said:


> obsigna
> 
> I will try it. Anyway I want to add just the lists I find on uBlock Origin that are not already on void-zones-tools - _at least for now_.
> 
> The Opera integrated adblock use (IIRC) Easylist what is also on uBlock Origin, but the integrated adblock is faster than the plugin.



Do these external lists exist in the Hosts file format (either of 127.0.0.1 or 0.0.0.0 will do)? If yes, then simply execute: `cat Easylist_in_hosts_format >> /usr/local/etc/void-zones/my_void_hosts.txt`

If no, please sent me a sample, so I can check it for a possible conversion.


----------



## obsigna (Nov 12, 2016)

MarcoB said:


> Afaics only place where a lot of ads are not blocked is on Linkedin. These are mostly "sponsored content". Is blocking this kind of ads possible with the tool? Or do we really need a browser plugin for this?


Can you identify distinguished domain names for these sponsored ads, i.e. ones that are different from the non-ad content on LinkedIn? For example, if the ads are coming from spads.linkedin.com and the non-ad content is served from www.linkedin.com, then you would simply add the entry 0.0.0.0 spads.linkedin.com to /usr/local/etc/void-zones/my_void_hosts.txt, and run either the updating script void-zones-update.sh or the simple conversion script from post #60) and the ads should have gone.

It is worth to note, that also the client machines maintain a DNS cache. So it might be well possible, that you don't see the void-zones being 100 % effective, because some domain names are still being resolved from the DNS cache of the client.

On my Mac's I run `sudo killall -HUP mDNSResponder` for cleaning the DNS caches, unfortunately I don't know the commands for other client systems.


----------



## MarcoB (Nov 12, 2016)

obsigna said:


> Can you identify distinguished domain names for these sponsored ads, i.e. ones that are different from the non-ad content on LinkedIn? For example, if the ads are coming from spads.linkedin.com and the non-ad content is served from www.linkedin.com, then you would simply add the entry 0.0.0.0 spads.linkedin.com to /usr/local/etc/void-zones/my_void_hosts.txt, and run either the updating script void-zones-update.sh or the simple conversion script from post #60) and the ads should have gone.


 Yeah I tried to find the domain names where those ads are coming from but couldn't find them. I'll keep searching then.



> It is worth to note, that also the client machines maintain a DNS cache. So it might be well possible, that you don't see the void-zones being 100 % effective, because some domain names are still being resolved from the DNS cache of the client.
> 
> On my Mac's I run `sudo killall -HUP mDNSResponder` for cleaning the DNS caches, unfortunately I don't know the commands for other client systems.


Don't know the command on FreeBSD either but will look for it. Thanks.


----------



## rigoletto@ (Nov 12, 2016)

obsigna said:


> Do these external lists exist in the Hosts file format (either of 127.0.0.1 or 0.0.0.0 will do)? If yes, then simply execute: `cat Easylist_in_hosts_format >> /usr/local/etc/void-zones/my_void_hosts.txt`
> 
> If no, please sent me a sample, so I can check it for a possible conversion.



I was looking on it right now, and unfortunately, apparently, some of the lists it use an AdBlock specific syntax. I will take o look on some of the websites uBlock get the lists to see if there is something more _normal_. Other lists appear to be ok.

All lists they use can be found here.

Thank you!

Some seem to have a very specific syntax what include the size of the banner (or something) to be removed.


----------



## rigoletto@ (Nov 12, 2016)

obsigna I could not find a way to send you a e-mail or a PM.

So, I opened all of their lists, and the only ones what seem will work without a major work are those:

http://hosts-file.net/.\ad_servers.txt
https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt
https://s3.amazonaws.com/lists.disconnect.me/simple_malware.txt
https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
http://mirror1.malwaredomains.com/files/justdomains
http://malwaredomains.lehigh.edu/files/immortal_domains.txt
http://someonewhocares.org/hosts/hosts  ### Already there
http://winhelp2002.mvps.org/hosts.txt   ### Already there
http://www.malwaredomainlist.com/hostslist/hosts.txt    ### Already there

One more, not in uBlock:

https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt

Thank you.


----------



## obsigna (Nov 13, 2016)

lebarondemerde said:


> obsigna I could not find a way to send you a e-mail or a PM.
> 
> So, I opened all of their lists, and the only ones what seem will work without a major work are those:
> 
> ...



Since, it is not a good idea to post e-mail addresses publicly, please may I ask you to pick it up on my BLog/Impressum.

I will look at the various lists in the next days. My main concern is not the format, but whether the lists are maintained.


----------



## obsigna (Nov 13, 2016)

Sr. Baron,

In the meantime, I had a look at the proposed lists, and furthermore I added the capability of scanning simple domain lists along side to listings in the Hosts file format to the hosts2zones tool, and the updates are on GitHub already.

http://hosts-file.net/.\ad_servers.txt

This file comes from the hpHosts site. I didn't include lists from hpHosts for 2 reasons.

These lists seem to be quite unbalanced, which becomes apparent by the mere number of total entries > 500000 hosts. This would not be a big problem for the hosts2zones tool, it would still process this in less than a second, however, the doubt remains if perhaps the major part of these lists consist of dead entries.


The License forbids automatic processing, so I do not suggest this by any means.
https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
https://s3.amazonaws.com/lists.disconnect.me/simple_malware.txt
https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt

These lists are said to be originated at Disconnect.me, (see: https://github.com/chrisaljoudi/uBlock/issues/1406), however, besides this claim, there is no further evidence. As a matter of fact, I stumbled across these lists already more than a year ago, and I also experimented with it. Nonetheless, I am hesitant to add these lists to the automatic processing scheme of the void-zones-tools, because of my doubts who owns and maintains the lists, and it is even not very clear whether these lists are actually maintained (only 300 more entries in more than one year).

http://mirror1.malwaredomains.com/files/justdomains
http://malwaredomains.lehigh.edu/files/immortal_domains.txt

These lists are actually from the same source, i.e. DNS-BH – Malware Domain Blocklist, which seems to be well maintained and supports inclusion into open source projects. I just added the justdomains list to the automatic updating scheme of my tools.

Finally, I facilitated inclusion of lists that are not part of my automated updating scheme. On invocation of hosts2zones by the shell script void-zones-update.sh, now 3 additional input files are passed:
x_void_list.txt, y_void_list.txt, z_void_list.txt.
With that in place it would be quite easy to include for example above ...lists.disconnect.me/simple_...txt files to the hosts2zones processing.

Simply execute the following command before updating the other zones:
`fetch -o - \
https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt \
https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt \
https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt \
https://s3.amazonaws.com/lists.disconnect.me/simple_malware.txt \
> /usr/local/etc/void-zones/x_void_list.txt`

Said command would place the respective lists joined together into /usr/local/etc/void-zones/x_void_list.txt, and that one would be converted/consolidated into the local-void.zones for filtering by Unbound on the next run of void-zones-update.sh. In the case these additional files are missing, the tool simply ignores these parameters.


----------



## rigoletto@ (Nov 13, 2016)

Thank you obsigna!

Already working here.


----------



## MarcoB (Nov 13, 2016)

Same here


----------



## wblock@ (Nov 14, 2016)

I don't understand this thread.  It started as Nvidia telemetry, then evolved into something that seemed to suggest you can block that with a hosts file.  The easy way to get past that is for the "telemetry" to just use static IP addresses.  There are numerous other ways it could be done through other channels.  Likewise with the Windows stuff.  As far as what it reports, a video card can potentially report every image displayed on your monitor.  Or the driver could do a little processing to grab text from the screen.


----------



## obsigna (Nov 14, 2016)

wblock@ said:


> I don't understand this thread.  It started as Nvidia telemetry, then evolved into something that seemed to suggest you can block that with a hosts file.


Seems you understood it subconsciously -- correct, once you have the facility in place, then add the telemetry domains, and you are almost done with it.


wblock@ said:


> The easy way to get past that is for the "telemetry" to just use static IP addresses.


Think about this again, keeping in mind that it is much easier to add a firewall rule than to keep domain lists updated:
`ipfw add 10 deny ip from any to nn.vv.id.ia`


wblock@ said:


> There are numerous other ways it could be done through other channels.


 Hardly, if the target system is controlling the DNS and the Firewall.


wblock@ said:


> Likewise with the Windows stuff.


During evolvement of this thread we found already a domain list of all the Windows 10 telemetry:

```
### Extra rules for @StevenBlack 's hosts project
### https://github.com/FadeMind/hosts.extras
### <Windows 10 Telemetry> < B E G I N >
0.0.0.0 a.ads1.msn.com
0.0.0.0 a.ads2.msads.net
0.0.0.0 a.ads2.msn.com
0.0.0.0 a.rad.msn.com
0.0.0.0 a-0001.a-msedge.net
0.0.0.0 a-0002.a-msedge.net
...
...
0.0.0.0 vortex-win.data.microsoft.com
0.0.0.0 watson.live.com
0.0.0.0 watson.microsoft.com
0.0.0.0 watson.ppe.telemetry.microsoft.com
0.0.0.0 watson.telemetry.microsoft.com
0.0.0.0 watson.telemetry.microsoft.com.nsatc.net
0.0.0.0 wes.df.telemetry.microsoft.com
0.0.0.0 win10.ipv6.microsoft.com
0.0.0.0 www.msftncsi.com
### <Windows 10 Telemetry> < E N D >
```



wblock@ said:


> As far as what it reports, a video card can potentially report every image displayed on your monitor.  Or the driver could do a little processing to grab text from the screen.



Yeah, people are not only concerned but developing and deploying already counter measures.

So, I guess, we are still on topic, aren't we?


----------



## wblock@ (Nov 14, 2016)

I understand that firewalls can block arbitrary addresses, but I also know that this would not stop anyone.  There is no reason a semi-random list of IP addresses cannot be used, and those would not necessarily be in a company's IP block.  Think "partners".  And of course, any update to the driver or firmware or even something that seems entirely unrelated can change it all entirely.  The point is that counting on DNS to stop this is very fragile.


----------



## obsigna (Nov 14, 2016)

wblock@ said:


> I understand that firewalls can block arbitrary addresses, but I also know that this would not stop anyone.



We are not talking about criminals, secret services, and intelligence agencies. We are talking about companies with tens to hundreds of millions of customers, to whom they want to keep in touch everyday. Just for the sake of load balancing, nowadays CDN services are used for this, that in turn assign domain aliases as entry points. Therefore, DNS walls are so effective these days.



wblock@ said:


> There is no reason a semi-random list of IP addresses cannot be used, and those would not necessarily be in a company's IP block.  Think "partners".



Maybe that some of these companies evaluate falling back to stone age methods once the share of their "customers" who uses telemetry blocking increases significantly, let's say from 0.01 ‰ to 25 % - Good luck!

Some of us may then think again about the suggestion of TeamBlackFox in message #21.



wblock@ said:


> And of course, any update to the driver or firmware or even something that seems entirely unrelated can change it all entirely.



Nice idea, changing the IP addresses for some customers who updated, and loosing contact to all the others. BTW, this is the reason why DNS exists.



wblock@ said:


> The point is that counting on DNS to stop this is very fragile.



I agree, in edge cases this method may be fragil, in 99.9 % of the usual cases it is effective, though.


----------



## obsigna (Nov 14, 2016)

The domain names, which NVIDIA seems to use for Telemetry (i.e. found in the calling home payload by others) might be:

```
gfe.nvidia.com
gfwsl.geforce.com
```
I do not own a GEFORCE, and cannot check this. Before adding this to the void-zones, it might be good to wait for an independent confirmation.


----------



## wblock@ (Nov 14, 2016)

obsigna said:


> We are not talking about criminals, secret services, and intelligence agencies. We are talking about companies with tens to hundreds of millions of customers


It is hard to see a big difference between those.  None are greatly concerned with laws, all want to conceal their activities.  Let me just put it this way: blocking with DNS assumes that the people who want their stuff to report home do not have a big incentive to try alternate methods.


----------



## rhsbsd (Nov 27, 2016)

It doesn't stop there. I'm sure most of you know about this https://newrepublic.com/article/117037/us-gives-iana-and-dns-control-icann. That's just one article, one view point. Recently on my own personal machine I decided to enable `unbound` and make it permanent. Its probably the best thing I've ever done because it allows you to control with whom you will get your telephone information from and spread it out over those (I use 18) dns servers. Then you can run `dnstop -l 4 wlan0` all the time and this allows you to find out exactly who is calling whom. This then in turn enables you to fix the offending application by whatever means you prefer. I keep, successfully, d-tuning so to speak, my choice of web browser but still maintain full functionality. It stays that way until I choose to update it and then once again plug the leaks.
Going after the application is much better than constantly trying to shore up, or build a defense. Also, if you contact the sofware vendors involved, and more people start doing that, you may be in for a pleasant surprise. FreeBSD is the best choice for doing all of the above and then some.


----------

