FLOSS Project Planets

OpenSource.com: Spark ignites at ApacheCon

Planet Apache - Mon, 2015-04-06 05:00

ApacheCon is coming up, and within that massive conference there will be a glimmering gem: a forum dedicated to Spark. The Spark Forum will have speakers from the Hive project, the Pig project, and the Sqoop project. Plus, two talks about Spark Streaming—one will be introductory, and the other developer-focused—and one about real-world data science using Spark. The forum will be an opportunity to see the progress made and share thoughts on the roadmaps of the projects.

Categories: FLOSS Project Planets

Plasma 5 live images for openSUSE and on the default openSUSE desktop

Planet KDE - Mon, 2015-04-06 04:35

A lot has been happening on the KDE side of openSUSE… this post summarizes what’s been going on so far.

Live media for Plasma 5

One of the most-often requested ways to test Plasma 5, given it can’t be coinstalled with the 4.x Workspace, is the availability of live images to test either in VM or bare metal without touching existing systems.

Given that other distributions started doing so since a while, naturally openSUSE couldn’t stay still. ;) Thanks to the efforts of Hrvoje “shumski” Senjan, we have now live media available for testing out Plasma 5!

  • Download location: the ISO file you’re looking for is called openSUSE-Plasma5 (currently x86_64 only)

The image is based on the current Tumbleweed and takes the latest code from git. If you test this in a virtual machine, bear in mind that there are some issues with VirtualBox and Plasma 5, and that QtQuick’s reliance on openGL can cause problems in general with virtual machines.

And if you find a bug… if it’s in the core distribution, or in the KDE packaging, head over to openSUSE’s Bugzilla. If it’s instead in the software, the KDE bug tracker is your friend.

Questions? Head over to the opensuse-kde ML, or the #opensuse-kde channel on Freenode.

Plasma 5 as default in openSUSE

You may have read on a recent Softpedia article that Plasma 5 is going to become the default in openSUSE. That’s correct (what did you expect, a retraction? ;): I and the others of the team (Raymond and Hrvoje) have been using Plasma 5 for a long time, not only because we like to stay on the bleeding edge ;) but also to see how it would fare for openSUSE. In the mean time, we reported bugs, sometimes fixed them, and occasionally landed one or two features in.

With the upcoming Plasma 5.3, we feel that it is of the level of quality expected from the default openSUSE desktop, and therefore we have set up preparations for the switch. As Rome wasn’t built in a day, it won’t happen straight away ;) but it will involve changes in the repositories and in the packaging, which are summarized below:

  • We will start the migration around the end of April, as long as the basic openQA tests are ready;
  • We will release KDE Applications 15.04 in Tumbleweed at the same time;
  • The KF5 ports of the applications present in KDE Applications 15.04 will obsolete their existing counterparts (hence the KF5 version will replace the 4.x version).  The same will happen for the kdebase4-workspace and Workspace 4.x packages;
  • Afterwards, the 4.x Workspace will not be supported or maintained for Tumbleweed. Help from the community is welcome in case anyone wants to step up and maintain the packages.
  • The default menu applet will be Kicker (as opposed to Kickoff used in the 4.x tiems;
  • The default theme for Plasma 5 will be Breeze (the upstream default), and we will use the menu structure provided by upstream KDE (as opposed to the custom structure we use today);
  • The repository layout will change. We will have three repositories holding KDE software:
    • KDE:Frameworks – the KF5 libraries and Plasma 5;
    • KDE:Applications – KDE Applications releases
    • KDE:Extra – Other KDE/Qt related community packages.
  • For each of these repositories, there will be also an “unstable” variant, tracking the current git master state.

(The full IRC log of the last meeting outlines these points in detail)

There are still some points open for discussion, in particularly for the update applet: should we keep Apper? Would Muon be a drop-in replacement? Or are we better off without an applet at all?

Of course, input  and help from the community is welcome. Hop on IRC or on the ML (see above for where to look)  if you want to help and participate in this large transition.

Categories: FLOSS Project Planets

Jonathan Carter: Squashfs Performance Testing

Planet Debian - Mon, 2015-04-06 03:35
Experiments

Last week I discovered The Fan Club’s Experiments page. It reminds me of the Debian Experiments community on Google+. I like the concept of trying out all kinds of different things and reporting back to the community on how it works. I’ve been meaning to do more of that myself so this is me jumping in and reporting back on something I’ve been poking at this weekend.

Introducing Squashfs

Squashfs is a read-only compressed filesystem commonly used on embedded devices, Linux installation media and remote file systems (as is done in LTSP). Typically, a system like tmpfs, unionfs or aufs is mounted over this read-only system to make it usable as a root filesystem. It has plenty of other use cases too but for the purposes of this entry we’ll stick with those use cases in mind. It supports gzip, lzo and xz(lzma) as compression back-ends. It also supports block sizes from 4K up to 1M.

Compression technique as well as block size can have major effects on both performance and file size. In most cases the defaults will probably be sufficient, but if you want to find a good balance between performance and space saving, then you’ll need some more insight.

My Experiment: Test effects of squashfs compression methods and block sizes

I’m not the first person to have done some tests on squashfs performance and reported on it. Bernhard Wiedemann and the Squashfs LZMA project have posted some results before, and while very useful I want more information (especially compress/uncompress times). I was surprised to not find a more complete table elsewhere either. Even if such a table existed, I probably wouldn’t be satisfied with it. Each squashfs is different and it makes a big difference whether it contains already compressed information or largely uncompressed information like clear text. I’d rather be able to gather compression ratio/times for a specific image rather than for one that was used for testing purposes once-off.

So, I put together a quick script that takes a squashfs image, extracts it to tmpfs, and then re-compressing it using it all the specified compression techniques and block sizes… and then uncompressing those same images for their read speeds.

My Testing Environment

For this post, I will try out my script on the Ubuntu Desktop 14.04.2 LTS squashfs image. It’s a complex image that contains a large mix of different kinds of files. I’m extracting it to RAM since I want to avoid having disk performance as a significant factor. I’m compressing the data back to SSD and extracting from there for read speed tests. The SSD seems fast enough not to have any significant effect on the tests. If you have a slow storage, the results of the larger images (with smaller block sizes) may be skewed unfavourably.

As Bernhard mentioned on his post, testing the speed of your memory can also be useful, especially when testing on different kinds of systems and comparing the results:

# dd if=/dev/zero of=/dev/null bs=1M count=100000 104857600000 bytes (105 GB) copied, 4.90059 s, 21.4 GB/s

CPU is likely to be your biggest bottleneck by far when compressing. mksquashfs is SMP aware and will use all available cores by default. I’m testing this on a dual core Core i7 laptop with hyper-threading (so squashfs will use 4 threads) and with 16GB RAM apparently transferring around 21GB/s. The results of the squashfs testing script will differ greatly based on the CPU cores, core speed, memory speed and storage speed of the computer you’re running it on, so it shouldn’t come as a surprise if you get different results than I did. If you don’t have any significant bottleneck (like slow disks, slow CPU, running out of RAM, etc) then your results should more or less correspond in scale to mine for the same image.

How to Run It

Create a directory and place the filesystem you’d like to test as filesystem.squashfs, then:

$ apt-get install squashfs-tools $ wget https://raw.githubusercontent.com/highvoltage/squashfs-experiments/master/test-mksquashfs.sh $ bash test-mksquashfs.sh

With the default values in that file, you’ll end up with 18 squashfs images taking up about 18GB of disk space. I keep all the results for inspection, but I’ll probably adapt/fix the script to be more friendly to disk space usage some time.

You should see output that look something like this, with all the resulting data in the ./results directory.

* Setting up... - Testing gzip * Running a squashfs using compression gzip, blocksize 4096 * Running a squashfs using compression gzip, blocksize 8192 * Running a squashfs using compression gzip, blocksize 16384 ... - Testing lzo * Running a squashfs using compression lzo, blocksize 4096 * Running a squashfs using compression lzo, blocksize 8192 * Running a squashfs using compression lzo, blocksize 16384 ... - Testing xz * Running a squashfs using compression xz, blocksize 4096 * Running a squashfs using compression xz, blocksize 8192 * Running a squashfs using compression xz, blocksize 16384 ... * Testing uncompressing times... * Reading results/squashfs-gzip-131072.squashfs... * Reading results/squashfs-gzip-16384.squashfs... * Reading results/squashfs-gzip-32768.squashfs... ... * Cleaning up... On to the Results

The report script will output the results into CSV.

Here’s the table with my results. Ratio is percentage of the size of the original uncompressed data, CTIME and UTIME is compression time and uncompress time for the entire image.

Filename   Size Ratio CTIME UTIME squashfs-gzip-4096.squashfs   1137016 39.66% 0m46.167s 0m37.220s squashfs-gzip-8192.squashfs   1079596 37.67% 0m53.155s 0m35.508s squashfs-gzip-16384.squashfs   1039076 36.27% 1m9.558s 0m26.988s squashfs-gzip-32768.squashfs   1008268 35.20% 1m30.056s 0m30.599s squashfs-gzip-65536.squashfs   987024 34.46% 1m51.281s 0m35.223s squashfs-gzip-131072.squashfs   975708 34.07% 1m59.663s 0m22.878s squashfs-gzip-262144.squashfs   970280 33.88% 2m13.246s 0m23.321s squashfs-gzip-524288.squashfs   967704 33.79% 2m11.515s 0m24.865s squashfs-gzip-1048576.squashfs   966580 33.75% 2m14.558s 0m28.029s squashfs-lzo-4096.squashfs   1286776 44.88% 1m36.025s 0m22.179s squashfs-lzo-8192.squashfs   1221920 42.64% 1m49.862s 0m21.690s squashfs-lzo-16384.squashfs   1170636 40.86% 2m5.008s 0m20.831s squashfs-lzo-32768.squashfs   1127432 39.36% 2m23.616s 0m20.597s squashfs-lzo-65536.squashfs   1092788 38.15% 2m48.817s 0m21.164s squashfs-lzo-131072.squashfs   1072208 37.43% 3m4.990s 0m20.563s squashfs-lzo-262144.squashfs   1062544 37.10% 3m26.816s 0m15.708s squashfs-lzo-524288.squashfs   1057780 36.93% 3m32.189s 0m16.166s squashfs-lzo-1048576.squashfs   1055532 36.85% 3m42.566s 0m17.507s squashfs-xz-4096.squashfs   1094880 38.19% 5m28.104s 2m21.373s squashfs-xz-8192.squashfs   1002876 34.99% 5m15.148s 2m1.780s squashfs-xz-16384.squashfs   937748 32.73% 5m11.683s 1m47.878s squashfs-xz-32768.squashfs   888908 31.03% 5m17.207s 1m43.399s squashfs-xz-65536.squashfs   852048 29.75% 5m27.819s 1m38.211s squashfs-xz-131072.squashfs   823216 28.74% 5m42.993s 1m29.708s squashfs-xz-262144.squashfs   799336 27.91% 6m30.575s 1m16.502s squashfs-xz-524288.squashfs   778140 27.17% 6m58.455s 1m20.234s squashfs-xz-1048576.squashfs   759244 26.51% 7m19.205s 1m28.721s

Some notes:

  • Even though images with larger block sizes uncompress faster as a whole, they may introduce more latency on live media since a whole block will need to be uncompressed even if you’re just reading just 1 byte from a file.
  • Ubuntu uses gzip with a block size of 131072 bytes on it’s official images. If you’re doing a custom spin, you can get improved performance on live media by using a 16384 block size with a sacrifice of around 3% more squashfs image space.
  • I didn’t experiment with Xdict-size (dictionary size) for xz compression yet, might be worth while sacrificing some memory for better performance / compression ratio.
  • I also want stats for random byte reads on a squashfs, and typical per-block decompression for compressed and uncompressed files. That will give better insights on what might work best on embedded devices, live environments and netboot images (the above table is more useful for large complete reads, which is useful for installer images but not much else), but that will have to wait for another day.
  • In the meantime, please experiment on your own images and feel free to submit patches.
Categories: FLOSS Project Planets

IPv6 – Set Up An IPv6 LAN with Linux

LinuxPlanet - Sun, 2015-04-05 12:05

Setting up an IPv6 LAN with Linux? Ever wonder how to do that? For years we have heard the dire predictions about the impending doom of IPv4 and the imminent arrival of IPv6. As with any eschatological  predictions you either choose to ignore it and hope for the best, or you could prepare for the event as best one can. So far the former strategy has served many sysadmins well and prooved to be an effective strategy .

If however, you decided to gird your loins and face IPv6 head on, you probably quickly discovered, that although there is a lot out there about the theory of IPv6 there is very little in the way of practical how tos when it comes to setting up an IPv6 LAN.  What makes understanding IPv6  troublesome is the complexity of working in a mixed environment of IPv4 and IPv6. This complexity becomes evident when one tries to connect to an external IPv6 network or the Internet which is still predominantly IPv4.

Steps to Set Up an IPv6 LAN

This blog posts break this down into two separate problems:

  1. Seting up  an IPv6 LAN network with Linux,
  2. Connecting your IPv6 network to the Internet

If you separate these two issues out its much easier to figure out what you need to do. Both of these steps have issues that need to be understood before the IPv6 "ah-hah" moment. Once that happens you will also have the "oh no" moment which might help you understand why there is such slow movement on IPv6 adoption.

In the first  part we will configure an Ubuntu 14.10 server to manage an  IPv6 LAN. In the 2nd part  we will deal with the myriad of options to connect an IPv6 network to the internet.

IPv6 Addressing - Some Theory

First we need to cover some theory on IPv6 addresses. There is a lot of article covering IPv6 addressing on the web, so I will just summarize what you need to know to proceed with setting up an IPv6 network. There are some nuances and subtleties we will brush over to provide you with a working conceptual model.

  • IPv6 addresses consist of 8 groups of 16 bit hexadecimal numbers to give a total address of 128 bits.  (See Global addresses below for explanation of the 2001:0db8::/32 address block.)
    • 2001:0db8:85a3:0000:0000:8a2e:0370:7334
    • 2001:db8:85a3:0:0:8a2e:370:7334 -> leading zeros (0) are dropped and in the case of a group of zeros (0000) it is reduced to just 0,
    • 2001:db8:85a3::8a2e:370:7334 -> lastly consecutive zeros are simply replaced with an empty double colon ::
  • The first 4 group of hexadecimal numbers of an address, 64 bits of the 128 bits,  is the network prefix (network mask). All IPv6 networks have a 64 bit network prefix,
  • The remaining 64 bits are the host identifier,

Sometime you will see an addresses listed with a prefix such as /48 or /56 etc. This does not mean that 16 (64-48) or 8 (64-56) bits of the 64 network prefix has been reserved for use by hosts as with IPv4 CIDR. The network address is always 64 bits long.

This notation refers to a block of networks. i.e all networks that begin with the first 48 or 56 bits set as specified. This is known as a routing prefix  and is used in routing rules, resulting in smaller routing tables. It is also used for when you are assigned a block of IPv6 networks.

The idea with IPv6 is that you should be assigned a block of networks by your ISP or IANA instead of a single host address or single IPv4 network as currently happens with IPv4.

The remaining bits of the network prefix 16 (64-48) or 8 (64-56) are called the subnet id. So the routing prefix +  subnet id make up the network prefix of an IPv6 address.  Dont' be confused by the use of the word subnet in subnet id.It is not an IPv4 subnet mask. It is simple the part of the network prefix you get to assign yourself as the administrator of that block of network addresses. 

So if you get an IPv6 address block with a 56 bit routing prefix it means you can have 255 (28)  networks each with  1.844674407×10¹⁹ (264) hosts!. Its up to you to determine how the subnet portion is used to create the network address. So if you are given a block of IPv6 network such as fdc8:282a:f54c::/48 it means you can have 216 networks. Your networks addresses are :

  • fdc8:282a:f54c:1::/64 
  • fdc8:282a:f54c:2:/64
  • ... 
  • fdc8:282a:f54c:ffff:/64

We wil come to the host identifier portion later later. The IPv6 network address space has been "sliced up" into different blocks. What you need to know about these blocks is given below: (each address block is explained further later)

Special IPv6 Address Blocks Name Prefix Explanation Link Local fe80::/10 Although this routing prefix is only 10 bits leaving 54 bits for up to 254 networks only one subnet id has been allocated so far by the specification which is fe80:0:0:0 or  fe80::/64 Unique Local Addresses(ULA) fc00::/7 Although this routing prefix is only 7 bits, the 8th bit must always be 1 according to the spec. So what you will see in practice is fd00::/7. At some later point we may see fc00::/7. We will be using this address block in our setup. Global Addresses  2001::/23  Global addresses will in fact be most of the address space of IPv6  So far the 2001::/23 block has been assinged and this is what you are likely to see in practice until further blocks are assigned to regional registrars. Within this some addresses have been reserved for a special purpose such as 2001:0db8::/32 which is reserved for documentation so if anyone copies it it won't actually route. To see what block have been assigned see the IANA site. 

 
For more information on the address blocks see the IANA site

Set Up an IPv6 LAN with Linux

We will set up an IPv6 network incrementally. We will start with the simplest and most trivial IPv6 and add services as we go. This will help us arrive at a understanding of how the various services fit together. We will go from the simplest IPv6 network to one which has all the basic network services required of a business network.

Simplest IPv6 Network - Link Local Only

To setup the simplest IPv6 network you just have to boot up a host or two with a IPv6 enabled operating system such as Ubuntu. Open a terminal and type:

"ip -6 address list"

You should see output similar to the following:

1: lo: mtu 65536 ,up,lower_up>
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qlen 1000
inet6 fe80::922b:34ff:fe7b:6ff1/64 scope link
valid_lft forever preferred_lft forever ,multicast,up,lower_up>,up,lower_up>

IPv6 link local  addresses have been assigned automatically to any interfaces that you have. The IPv6 localhost address (IPv4 127.0.0.1) is ::1/128. You can do the same on another host to gets it IPv6 link local address and then do a IPv6 ping with "ping6" - note the 6.

"ping6 fe80::922b:34ff:fe7b:6ff1"

The fe80::/64 network prefix is the link local network as explained in the table above. It should be the only IPv6 network address you will see across different physical networks. In fact every host on an IPv6 network must have an link local address (fe80::/64).

Host Identifier Generation

The host identifier portion of the link local address, the remaining 64 bits, is generated from the mac address with a algorithm applied to extend the 48 bit mac address to the 64 bit host address required for IPv6. See EUI64 for the algorithm used. The host identifier may also be manually assigned by the system administrator. This introduces the risk of duplicate IP addresses being assigned, so IPv6 has a duplicate address detection protocol that allows hosts to determine if there is a conflict before assigning itself an address.

In most cases you will let this be automatically generated. In IPv4 initially IP address had to be manually assigned or assigned via a DHCP server. Later the 169.254/16 address range was reserved for auto-configuration in IPv4 network. Unlike IPv4 your interfaces will always have an fe80::64 address, it is not used instead of a valid IPv6 address. In IPv6 your interfaces will typically have multiple IP addresses. 

Why do you need a Link Local Address?

IPv6 configuration is done using layer 3 (network layer) protocols and not layer 2 (media layer eg. Ethernet) as with IPv4; so a valid IPv6 address is required before any additional configuration can be done. Of couese it also allows for zero config simple networks.

Pros and Cons of Link Local Network

With a link local address you can communicate with other IPv6 hosts on the local network segment or broadcast domain. i.e the same switch or shared media network. So for a home LAN not connected to the internet this is all that is required. You can connect to your printer, Smart TV, PlayStation etc automatically using protocols such as UPnP and multicast DNS (ZeroConf).  Connecting to the internet, or a network in a different physical network or logical network, will require a bit more work.

Set Up A Routable IPv6 Network

If this all there was too it then we could all go home. But if you start to think about it you will begin to have some doubts as to how useful a link local only IPv6 network is.

  • How do I assigned the same address to a host every time without doing it manually? (it is possible for a nodes host identifier to change between reboot if there is a conflict) 
  • What if I change the NIC and get a different IP address?
  • How do I configure the hosts for routes and other services such as DNS, NTP etc?
  • How do I communicate between two internal networks separated by a router or WAN link?

To address these issues you need to assign yourself a non-site local address. This can be a unique local address (ula) or a global address. For a global address you will need to get an IPv6 network block from your ISP or get one assigned to you by IANA. So we will make use of a ULA address which you can assign yourself.

What is the difference between a ULA and a Global address?  

By convention a ULA is not routed over the public internet. Routers on the public IPv6 network should refuse to route such traffic in a similar manner to private IPv4 addresses. Essentially there should be no routing entries in the routers responsible for internet traffic, making them unreachable from outside an organisation.

If you are going to start experimenting with IPv6 there are two reasons to use a ULA

  •  you should start with a ULA address to avoid any mis-configuration disasters.
  • It might be hard to get a global IPv6 address assigned to you. There are very few ISP handing out IPv6 network addresses currently so in some cases its the only choice available to you.
Unique Local Addresses

One feature of unique local addresses is that they should be different for every network you see. Unlike IPv4 where private addresses (196.128/16, 10/8 and 172.16/16) meant there are often networks with the same network mask - eg nearly every home and office has a  network with the network mask of 192.168.1.0/24 address or 10.0.0.0/24 range; you might never see a duplicate IPv6 ULA network address. This is because only the first 8 bits of the network prefix are fixed at "fd". The remaining 56 bits of the netowrk prefix, the subnet id, can be randomly selected. System administrators are meant to create the subnet id themselves. A handy way to generate the subnet id for the ULA is to use a site like unique-local-ipv6.com. From here you will get a /48 address range meaning you can have up to 65356 private networks!

Its generally a good idea to use a random subnet id rather than generate one like fd01:1:1:1::0/64 as this increase your chance of a conflict. Why would you be worried about a conflict if these are not routable? Have you ever had to merge two network that had the same IPv4 address range? Have you ever tried to setup a VPN between two network with the same IP network range?

Global Addresses

Global address will be assigned to you by an ISP unless you get your own block and tell your ISP to route it to you. So much like you get a public IP address from your ISP for IPv4 you will in future, get an IPv6 network address range when you dial up. Note: not a single IP address but a whole block of IPv6 addresses. Depending on your ISP you may get only one network or be assigned a block with multiple networks. In this case the router will received the network address prefix to use on your network. It will work the same as for the steps below except instead of a ULA network it will be a global address. Note you don't get assigned a full IPv6 address. You get the network prefix.

So to summarise. You will need at least two  IPv6 addresses for each interface if you want to do normal networkng tasks like route between network. A link local which is always present and at least one ULA or global address or perhaps all three!

For our exercise we will use ULA addresses to setup an IPv6 only LAN.

Set Up an ULA IPv6 Network in Linux

You can setup the following on a network that is already configured for IPv4. You can run IPv6 in parrallel with IPv4. This is known as a dual stack setup. Once done you can stop the IPv4 services and run the network on IPv6 only or keep it dual stack. One reason to test without IPv4 infrastructure running is to convince yourself that your network really is working over IPv6.

Ok, now that some theory is out the way on to how to practically setup an IPv6 only LAN. First we need some more theory :( We have already seen how a link local address is assigned. But how is the ULA network prefix assigned and how are default routes set up? For this IPv6 makes use of a router advertisment service that runs on the local network router. Once an link local address has been assinged the host will ask (solicit) any routers for configuration information. The router response with a router advertisment. This advertisment contains the ULA prefix and the address of the router for the default route. Initially the router did not provide a DNS address but this was later added so a node now has an ULA IPv6 address and a default gateway and all should be good. One thing to note here is the host is not provided with an IPv6 address. It is just provided with the network prefix and its up to the host to generate the host portion of the address. This is known as stateless address assignmetn. The router does not assign an address per se. it has no idea what address have been assigned to hosts. Hence the stateless in the term stateless automatic address configuration (SLAAC)


First assign the router a static IPv6 address from the ULA network:

sudo vi /etc/network/interfaces

auto eth0
iface eth0 inet6 static
   address fd5d:12c9:2201:1::1
   netmask 64
   autoconf 0
   dad-attempts 0
   accept_ra 0

This sets up a static IPv6 address in our range for our Linux router.

sudo apt-get install radvd

vi /etc/radvd.conf

interface eth0
{
        AdvSendAdvert on;
        prefix fd5d:12c9:2201:1::1/64 {
                AdvOnLink on;
                AdvAutonomous on;
        };
    #Send DNS Server setting if not using Dynamic DNS updates
    RDNSS fd5d:12c9:2201:1::2{
    };
};

Restart the service and then on a client restart the network. You should see two IPv6 address on your network card. "ip -6 address list". You can ping the router with the ping6 utility:

"ping6 fd5d:12c9:2201:1::1" if this doesn't work try "ping6 fd5d:12c9:2201:1::1 -I eth0" -> Use the interface with the assigned IPv6 address

Congratulations you have an IPv6 network up and running.

Step 2

Now several questions arise:

1) What is I want to send down other configuration information such as the NTP or SMTP server settings?
2) What is I want to make sure the same IPv6 address always get assigned to a server like the NTP or SMPT server?
3) What is I want to track IP address assignement?
4) What if I want to provide dynamic updates to the local DNS server?

To use DHCP you need to configure the radvd service to tell all nodes to contact a DHCP server. You can configure radvd to tell the nodes to contact the DHCP server for configuration info only or to get configuration information and IP address from the DHCP server. We will use DHCP to send configuration information such as DNS servers and to assign IP addresses:

interface eth0
{
        AdvSendAdvert on;
        prefix fd5d:12c9:2201:1::1/64 {
                AdvOnLink on;
                AdvAutonomous on;
                AdvManagedFlag on; # get a full IP address from the DHCP server
                AdvOtherConfigFlag on; # get other configuration info from the DHCP server
        };
};

 

Setting up DHCP6 is similar to DHCP for IPv4. We will use the isc-dhcpd-server

"apt-get install isc-dhcp-server"


Create a file /etc/dhcpd/dhcpd6.conf with the following entries:


ddns-update-style interim;
ddns-updates on;

update-conflict-detection false;
update-optimization false;

option domain-name "jumpingbean.co.za";
option dhcp6.name-servers fd5d:12c9:2201:1::2;

default-lease-time 600;
max-lease-time 7200;
include "/etc/dhcp/rndc.key";

log-facility local7;

zone jumpingbean.co.za. {
        primary 127.0.0.1;
        key rndc-key;
}


zone 1.0.0.0.1.0.2.2.c.9.2.1.d.5.d.f {
        primary 127.0.0.1;
        key rndc-key;
}


subnet6 fd5d:12c9:2201:1::/64 {
        range6 fd5d:12c9:2201:1::100 fd5d:12c9:2201:1::200;
}


Here we setup the DHCP server to provide the DNS server and also specify which Zone file should be updated in the local DNS server when an address is assigned. These are the "zone" entries about and play the same role as zone entries when you configure IPv4 dhcp server. We also set the DNS server and domain name to use on nodes. Note you can run an IPv4 and IPv6 DHCP server in parallel as they listen on different ports.

Note: To setup a fixed IPv6 address in DHCPv6 you make use of a DUID (Device Unique ID) which is not the mac address which is used for IPv4 DHCP. The DUID is assinged by the operating system and remains the same even if network cards change. 

host example {
  host-identifier option dhcp6.client-id 31:30:30:30:30:31:33;
  fixed-address6 fd5d:12c9:2201:1::101;
}

I am not aware of a way to get the DUID in Linux other than by looking in the leases file on the DHCP server. A binary copy of the node DUID can be found at /var/lib/dhcpv6/dhcp6s_duid. If anyone knows how to read this file please let the internet know :)

To start the isc-dhcp server with the option "-6". You can set this in /etc/defaults/isc-dhcpd-server "OPTIONS="-6". On Ubuntu 14.10 this is ignored and it start in IPv4 mode. To get the dhcp server to start in DHCPv6 mode add this to the /etc/rc.local file as a temporary solution.

dhcpd -6  -cf /etc/dhcp/dhcpd6.conf -lf /var/lib/dhcp/dhcpd6.leases eth0

You might also have apparmour complain about the lease file if you try and write it to a different location. You can either stop apparmor or configure the dhcp server to write to location that its profile supports writing to.


/etc/bind/named.conf.options

options {
        directory "/var/cache/bind";

        // If there is a firewall between you and nameservers you want
        // to talk to, you may need to fix the firewall to allow multiple
        // ports to talk.  See http://www.kb.cert.org/vuls/id/800113

        // If your ISP provided one or more IP addresses for stable 
        // nameservers, you probably want to use them as forwarders.  
        // Uncomment the following block, and insert the addresses replacing 
        // the all-0's placeholder.

         forwarders {
                8.8.8.8;
                4.4.4.4;
         };

        dns64 fd5d:12c9:2201:1:1:1::/96 {
                clients {
                        any;
                };

                exclude {
                        any;
                };
        };

        //========================================================================
        // If BIND logs error messages about the root key being expired,
        // you will need to update your keys.  See https://www.isc.org/bind-keys
        //========================================================================
        dnssec-validation auto;

        auth-nxdomain no;    # conform to RFC1035
        listen-on-v6 { any; };

};


/etc/named/named.conf.local

//
// Do any local configuration here
//

// Consider adding the 1918 zones here, if they are not used in your
// organization
//include "/etc/bind/zones.rfc1918";
include "/etc/bind/rndc.key";

zone "jozilug.co.za" {
        type master;
        allow-update { key rndc-key; };
        file "/var/lib/bind/jozilug.co.za";
};


zone "1.0.0.0.1.0.2.2.9.c.2.1.d.5.d.f.ip6.arpa" {
        type master;
        file "/var/lib/bind/fd5d:129c:2201:1";
        allow-update { key rndc-key; };
};

 

/var/lib/bind/jozilug.co.za

Zone file

$ORIGIN .
$TTL 604800     ; 1 week
jozilug.co.za           IN SOA  jozilug.co.za. admin.jozilug.co.za. (
                                182        ; serial
                                604800     ; refresh (1 week)
                                86400      ; retry (1 day)
                                2419200    ; expire (4 weeks)
                                604800     ; minimum (1 week)
                                )
                        NS      ns.jozilug.co.za.
                        A       127.0.0.1
                        AAAA    ::1
$ORIGIN jozilug.co.za.
$TTL 300        ; 5 minutes
android-a74e95670198fd6a A      10.0.10.4
                        TXT     "0002ec64161ce51591018b9eb0a01ae6b9"
$TTL 604800     ; 1 week
gateway                 AAAA    fd5d:12c9:2201:1::2
ns                      AAAA    fd5d:12c9:2201:1::2
$TTL 300        ; 5 minutes
trinity                 A       10.0.10.3
$TTL 187        ; 3 minutes 7 seconds
                        TXT     "025c83d7b0b5ca62d26381f057fbeed483"


/var/lib/bind/fd5d:129c:2201:1

;
; BIND reverse data file for broadcast zone
;
$TTL    604800
@       IN      SOA     ns.jozilug.co.za. admin.jozilug.co.za (
                              1         ; Serial
                         604800         ; Refresh
                          86400         ; Retry
                        2419200         ; Expire
                         604800 )       ; Negative Cache TTL
;
@       IN      NS      ns.jozilug.co.za.       

2.0.0.0.0.0.0.0.0.0.0.0.1.0.0.0.1.0.2.2.9.c.2.1.d.5.d.f.ip6.arpa.       IN      PTR     ns.jozilug.co.za


After all of this you will now have a fully functioning IPv6 LAN. We can now move to the 2nd problem how do I connect to the internet? At the time of writing this article the most common occurrence is for an ISP to assign a single IPv4 address to your router. As IPv6 becomes more widely adopted this will change and the step below will not be relevant. The issue is that IPv6 services cannot be access via an IPv4 address and IPv4 services cannot be natively connected from an IPv6 only host. So there is a transition pahse where both protocols will need to be present or a kludge will need to be used to convert from IPv4 to IPv6 and vica-versa. For the sake of simplicity we will assume an IPv4 address on the router and assume, at this point you only need to connect to IPv4 hosts on the internet. To enable an IPv6 only network to connect to the IPv4 network we can make use of nat64. This protocol converts all IPv6 addresses to a pool of IPv4 addresses and then routes the request to the ISP. This means there will be double natting which is not ideal.

Alternatively you can keep a dual stack and then setup a IPv6 tunnel at one of the well known tunnel brokers such as Hurricane Electric.

Categories: FLOSS Project Planets

Network Tunnels with Linux

LinuxPlanet - Fri, 2015-04-03 12:22

Creating network tunnels in Linux is easy and provides a great solution to what might otherwise be difficult to solve networking problems. Most people are familiar with tunnels as they are used to create virtual private networks (VPN) to connect remote sites or remote hosts to a central site securely, allowing the remote network to be accessed as if it is local.  

Linux Tunnels of Love - Tunnelling in Linux 16th Feb 2015 from Jumping Bean

 

Using Linux's IPRoute2 Utilities to create tunnels

In Linux we make use of the IPRoute2 utilities to create tunnels and to manage the,. Although you can use ifconfig to managed tunnels this is an aging and no longer adequate utility to manage networks. The ip utility can create gre or ipip (ip in ip) tunnels for both IPv4 and IPv6 networks. These tunnels are not natively encrypted so any encryption would need to be done at the layer 7 level to avoid eave-dropping. Of course you can also create secure tunnels easily using SSH as a SOCK proxy but this requires application level support to use the tunnel so not all traffic will go over the SOCK tunnel.

Creating tunnels involves three steps which must be performed on the two hosts that are participating in the tunnel:

  1. Create a logical tunnel device,
  2. Assign an IP address to the device
  3. Set up routing rules to route traffic over the tunnel

Each host is the mirror of the other when it comes to creating rules. E.G. The remote ip for host 1 is the local ip for host 2 and visa-versa.

1. Create the logical tunnel interface. The type of tunnel will depend on your needs either ipip, gre.sit etc.

"ip tunnel add tun0 mode ipip local [hosts  public ip] remote [remote hosts public ip]"
eg: "ip tunnel add tun0 mode ipip local 105.105.105.105 remote 41.41.41.41"

2. Assign an ip to the tunnel interface

"ip address add [private ip address] dev tun0"
"ip address add 192.168.12.2 dev tun0"

This private ip address is usually separate from the network address of the local network. It is a network that is for the tunnel only. The remote tunnel should be assigned an different address in the same netowork as its counterpart.

3. Add routing rules to the route traffic over the tunnel:

"ip route add [remote network]/24 via [remote tunnels IP address]"
"ip route add 10.0.10.0/24 via 192.168.12.2"

The above assumes that the remote host is on a network with a network address of 10.0.10/24. The local network could for example be 172.16.16/24 for example. The remote host will need to add a route for this network through the tunnel.

Uses for unencrypted tunnels

These days it generally not good practice to send anything out unencrypted. but you still might find a use for ipip or gre tunnels. They are a great way to bond ADSL connections or to create a seamless virtual network between hosts in different data centres and an absolute necessity to connect IPv6 networks swimming the sea of IPv4 networks.

Need Linux support and consulting or Linux training?  Why not contact us.

 

Categories: FLOSS Project Planets

Do Something!

LinuxPlanet - Thu, 2015-04-02 22:45
I'm a very opinionated person and I don't think that there is much I can do about that. Most of the time I try to not force my personal opinions on people, some of my friends and family might disagree, but I do honestly try. I like to think most people arrive at their opinions honestly and they represent a perspective, however different than mine, that is informed by things I might not be able to understand. I do know that my opinions on things have changed or maybe even evolved with time and I'd like to think we are all on a path headed towards our dreams. Maybe at different points on the path but still on a path. If I can help someone down the path with me, I try to do it. What I won't do is push someone to make ground on something by force.

In my own head I don't think I have a single personal philosophy that guides my life. Most of the time I feel like I'm drowning in my own self doubt. However, I do get put into the position of offering advice on peoples lives more than I'm comfortable with. Most of the time I just try my best to nudge people in a positive direction.

Lately however, I've been giving more and more thought to what I would call my personal brand of guiding wisdom. Now I obviously don't have the answer to eternal happiness, world peace or even how to not annoy the crap out of everyone by accident. The reality is, I'm pretty useless at making other peoples lives better most of the time, despite my grand ideas for changing the world.

What I do know is that when I'm at my most depressed or discouraged that I can always dig myself out. Even if it feels at the time like I never will. I don't have a magic silver bullet but I do know that every day I can chose to do at least one thing that makes my life or the life of those around me better and I think that mostly sums up my approach. As I've thought about it, I've boiled it down to something fairly concise.

"Do Something"

What I mean by that is you might not be able to control everything that happens to you and you also might not be able to control the way you feel about it. What you can do is move yourself down the path. Sometimes it's a moon surface leap and sometimes it's crawling through glass, but progress is progress. No, this won't guarantee your bills will get paid, you will save your marriage or heal a childhood pain. It might not even make you feel better. What it will do is put you a little closer, bit by bit.

If you are like me, most things feel overwhelming. I can be pretty hard on myself. I once told someone, "You can't say anything to me more hurtful than what I've said to myself." I think it might be one of the most honest things I've ever said. What I have found though that helps me more than anything, is doing something. Anything. As long as it's a positive step in the right direction. Even if it's just one small step with a million more to go, it's one step closer to my final destination.

No matter how small the gesture it can at least help you get into a better head space. It could be something for yourself, like getting chores you've been avoiding knocked out or something huge like finally telling someone how you care about them. You don't even have to do it for yourself. Sometimes when I'm at my lowest it helps to think about the things I wish others where doing for me at that moment and do it for someone else. One example is, for my own narcissistic reasons, I really like things I post to social media to get liked by my friends and family. Sometimes a post that I feel really strongly about or connected to will get almost completely ignored and it will send me into a tailspin of self doubt. In all likely hood there are multitudes of reasons people didn't take the time to click "like", and most are probably not related to me or my personal feelings. So, even in this silliest of first world problem situations, I try to reach out to others, click like on things my friends post or leave a positive comment. I would never do this disingenuously. I'm always clicking like or give a positive comment to something I actually like. I'm just trying to go a little more out of the way to make someone else feel good.

Now, does this achieve anything measurable. Most of the time no. Most of my friends are likely unaware I do this. Does it suddenly make all my neurotic obsession over whether people like me go away? not at all. What it does though is put me at least half a step closer to feeling better and more often than not it's enough to give me a clear head to see the next step I need to take. Sometimes that next step is one of those moon surface leaps that I can't believe I didn't take before.

Don't get me wrong, I don't hinge my day to day feelings on these silly little acts. Mostly I've learned about myself that I really like the feeling of creating something so I try to focus on those kinds of activities. I have loads of hobbies and things that I do that keep me moving forward. I think those count too. What I try not to do is sit around and think of all the things I should be doing and know for sure I won't do. I'd rather focus on the things I can do than the things I can't.

So now I think I can feel a tiny bit more comfortable in offering someone advice. Just "Do Something." As long as it's positive progress, it's worth it. No matter your situation, you can at least do something to make it better. No matter how insignificant it might seem at the time. I even keep a small daily journal where I try to write down the positive things I did that day. I also write some of the negatives but as long as there is at least one positive, it helps.

So?!?!

Do Something!

That's the best I've got.
Categories: FLOSS Project Planets

Bad Voltage Season 1 Episode 39: Ambitious but Rubbish

LinuxPlanet - Thu, 2015-04-02 10:57

Jono Bacon, Stuart Langridge and myself present Bad Voltage (without Bryan Lunduke who is currently struggling with an attack of Ebola), in which everything needs to be an order of magnitude better. Featuring flying bags of flammable liquid, 120 frames per second, and:

  • What needs to happen so that I can have a drone to deliver my pizza and pick up my shopping? Drawing a line through the technological, regulatory, and philosophical minefield standing between today and Jono’s Glorious Drone-Filled Future (2.40)
  • Tarus Balog speaks about OpenNMS, a network management system for big networks, and some recent changes in the project (27.01)
  • Jono reviews the Go Pro Hero 3+ silver edition extreme sports camera (45.24)
  • Jeremy Clarkson has been fired from Top Gear for hitting a colleague. We draw some fairly obvious parallels between the world of open source and this twin situation of standing up against unacceptable behaviour and whether a project is viable if a leading contributor is dismissed (59.38)

Listen to 1×39: Ambitious but Rubbish

As mentioned here, Bad Voltage is a project I’m proud to be a part of. From the Bad Voltage site: Every two weeks Bad Voltage delivers an amusing take on technology, Open Source, politics, music, and anything else we think is interesting, as well as interviews and reviews. Do note that Bad Voltage is in no way related to LinuxQuestions.org, and unlike LQ it will be decidedly NSFW. That said, head over to the Bad Voltage website, take a listen and let us know what you think.

–jeremy


Categories: FLOSS Project Planets
Syndicate content