FLOSS Project Planets

KnackForge: How to update Drupal 8 core?

Planet Drupal - Sat, 2018-03-24 01:01
How to update Drupal 8 core?

Let's see how to update your Drupal site between 8.x.x minor and patch versions. For example, from 8.1.2 to 8.1.3, or from 8.3.5 to 8.4.0. I hope this will help you.

  • If you are upgrading to Drupal version x.y.z

           x -> is known as the major version number

           y -> is known as the minor version number

           z -> is known as the patch version number.

Sat, 03/24/2018 - 10:31
Categories: FLOSS Project Planets

Petter Reinholdtsen: Easier recipe to observe the cell phones around you

Planet Debian - 3 hours 55 min ago

A little more than a month ago I wrote how to observe the SIM card ID (aka IMSI number) of mobile phones talking to nearby mobile phone base stations using Debian GNU/Linux and a cheap USB software defined radio, and thus being able to pinpoint the location of people and equipment (like cars and trains) with an accuracy of a few kilometer. Since then we have worked to make the procedure even simpler, and it is now possible to do this without any manual frequency tuning and without building your own packages.

The gr-gsm package is now included in Debian testing and unstable, and the IMSI-catcher code no longer require root access to fetch and decode the GSM data collected using gr-gsm.

Here is an updated recipe, using packages built by Debian and a git clone of two python scripts:

  1. Start with a Debian machine running the Buster version (aka testing).
  2. Run 'apt install gr-gsm python-numpy python-scipy python-scapy' as root to install required packages.
  3. Fetch the code decoding GSM packages using 'git clone github.com/Oros42/IMSI-catcher.git'.
  4. Insert USB software defined radio supported by GNU Radio.
  5. Enter the IMSI-catcher directory and run 'python scan-and-livemon' to locate the frequency of nearby base stations and start listening for GSM packages on one of them.
  6. Enter the IMSI-catcher directory and run 'python simple_IMSI-catcher.py' to display the collected information.

Note, due to a bug somewhere the scan-and-livemon program (actually its underlying program grgsm_scanner) do not work with the HackRF radio. It do work with RTL 8232 and other similar USB radio receivers you can get very cheaply (for example from ebay), so for now the solution is to scan using the RTL radio and only use HackRF for fetching GSM data.

As far as I can tell, a cell phone only show up on one of the frequencies at the time, so if you are going to track and count every cell phone around you, you need to listen to all the frequencies used. To listen to several frequencies, use the --numrecv argument to scan-and-livemon to use several receivers. Further, I am not sure if phones using 3G or 4G will show as talking GSM to base stations, so this approach might not see all phones around you. I typically see 0-400 IMSI numbers an hour when looking around where I live.

I've tried to run the scanner on a Raspberry Pi 2 and 3 running Debian Buster, but the grgsm_livemon_headless process seem to be too CPU intensive to keep up. When GNU Radio print 'O' to stdout, I am told there it is caused by a buffer overflow between the radio and GNU Radio, caused by the program being unable to read the GSM data fast enough. If you see a stream of 'O's from the terminal where you started scan-and-livemon, you need a give the process more CPU power. Perhaps someone are able to optimize the code to a point where it become possible to set up RPi3 based GSM sniffers? I tried using Raspbian instead of Debian, but there seem to be something wrong with GNU Radio on raspbian, causing glibc to abort().

Categories: FLOSS Project Planets

Iain R. Learmonth: Onion Services

Planet Debian - 4 hours 9 min ago

In the summer 2017 edition of 2600 magazine there is a brilliant article on running onion services as part of a series on censorship resistant services. Onion services provide privacy and security for readers above that which is possible through the use of HTTPS.

Since moving my website to Netlify, my onion service died as Netlify doesn’t provide automatic onion services (although they do offer automated Let’s Encrypt certificate provisioning). If anyone from Netlify is reading this, please consider adding a one-click onion service button next to the Let’s Encrypt button.

For now though, I have my onion service hosted elsewhere. I’ve got a regular onion service (version 2) and also now a next generation onion service (version 3). My setup works like this:

  • A cronjob polls my website’s git repository that contains a Hugo static site
  • Two versions of the site are built with different base URLs set in the Hugo configuration, one for the regular onion service domain and one for the next generation onion service domain
  • Apache is configured for two virtual hosts, one for each domain name
  • tor from the Debian archives is configured for the regular onion service
  • tor from git (to have next generation onion service support) is configured for the next generation onion service

The main piece of advice I have for anyone that would like to have an onion service version of their static website is to make sure that your static site generator is handling URLs for you and that your sources have relative URLs as far as possible. Hugo is great at this and most themes should be using the baseURL configuration parameter where appropriate.

There may be some room for improvement here in the polling process, perhaps this could be triggered by a webhook instead.

I’m not using HTTPS on these services as the HTTPS private key for the domain isn’t even controlled by me, it’s controlled by Netlify, so wouldn’t really be a great method of authentication and Tor already provides strong encryption and its own authentication through the URL of the onion service.

Of course, this means you need a secure way to get the URL, so here’s a PGP signed couple of URLs:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 As of 2017-09-23, the website at iain.learmonth.me is mirrored by me at the following onion addresses: w6d6vblb6vhuqxt6.onion tvin5bvfwew3ldttg5t6ynlif4t53y3mbmb7sgbyud7h5q6gblrpsnyd.onion This declaration was written and signed for publication in my blog. -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEfGEElJRPyB2mSFaW0hedW4oe0BEFAlnG1FMACgkQ0hedW4oe 0BGtTwgAp9PK6x1X9lnPLaeOOEALxn2BkDK5Q6PBt7OfnTh+f53oRrrxf0fmfNMH Qz/IDY+tULX3TZYbjDsuu+aDpk6YIdOnOzFpIYW9Qhm6jAsX4RDfn1cZoHg1IeM7 bCvrYHA5u753U3Mm+CsLbGihpYZE/FBdc/nE5S6LxYH83QZWLIW19EPeiBpBp3Hu VB6hUrDz3XU23dXn2U5/7faK7GKbC6TrBG/Z6dUtaXB62xgDIrPEMorwfsAZnWv4 3mAEsYJv9rnIyLbWamXDas8fJG04DOT+2C1NYmZ5CNJ4C7PKZuIYkaoVAp+pzLGJ 6BEBYaRvYIjd5g8xdVC3kmje6IM9cg== =lUvh -----END PGP SIGNATURE-----

Note: For the next generation onion service, I do currently have some logging enabled in the tor daemon as I’m running this service as an experiment to uncover any bugs that appear. There is no logging beyond the default for the version 2 hidden service’s tor daemon.

Another note: Current stable releases of Tor Browser do not support next generation onion services, you’ll have to grab an experimental build to try them out.

Viewing my next generation onion service in Tor Browser
Categories: FLOSS Project Planets

Iain R. Learmonth: Free Software Efforts (2017W38)

Planet Debian - 4 hours 9 min ago

Here’s my weekly report for week 38 of 2017. This week has not been a great week as I saw my primary development machine die in a spectacular reboot loop. Thanks to the wonderful community around Debian and free software (that if you’re reading this, you’re probably part of), I should be back up to speed soon. A replacement workstation is currently moving towards me and I’ve received a number of smaller donations that will go towards video converters and upgrades to get me back to full productivity.


I’ve prepared and tested backports for 3 packages in the tasktools packaging team: tasksh, bugwarrior and powerline-taskwarrior. Unfortunately I am not currently in the backports ACLs and so I can’t upload these but I’m hoping this to be resolved soon. Once these are uploaded, the latest upstream release for all packages in the tasktools team will be available either in the stable suite or in the stable backports suite.

In preparation for the shutdown of Alioth mailing lists, I’ve set up a new mailing list for the tasktools team and have already updated the maintainer fields for all the team’s packages in git. I’ve subscribed the old mailing list’s user to the new mailing list in DDPO so there will still be a comprehensive view there during the migration. I am currently in the process of reaching out to the admins of git.tasktools.org with a view to moving our git repositories there.

I’ve also continued to review the scapy package and have closed a couple more bugs that were already fixed in the latest upstream release but had been missed in the changelog.

Bugs closed (fixed/wontfix): #774962, #850570

Tor Project

I’ve deployed a small fix to an update from last week where the platform field on Atlas had been pulled across to the left column. It has now been returned to the right hand column and is not pushed down the page by long family lists.

I’ve been thinking about the merge of Compass functionality into a future Atlas and this is being tracked in #23517.

Tor Project has approved expenses (flights and hotel) for me to attend an in-person meeting of the Metrics Team. This meeting will occur in Berlin on the 28th September and I will write up a report detailing outcomes relevant to my work after the meeting. I have spent some time this week preparing for this meeting.

Bugs closed (fixed/wontfix): #22146, #22297, #23511


I believe it is important to be clear not only about the work I have already completed but also about the sustainability of this work into the future. I plan to include a short report on the current sustainability of my work in each weekly report.

The loss of my primary development machine was a setback, however, I have been donated a new workstation which should hopefully arrive soon. The hard drives in my NAS can now also be replaced as I have budget available for this now. I do not see any hardware failures being imminent at this time, however should they occur I would not have budget to replace hardware, I only have funds to replace the hardware that has already failed.

Categories: FLOSS Project Planets

Jeff Hinrichs: MicroPython on ESP32: Tools – virtual environments vs system installs

Planet Python - Sat, 2017-09-23 19:52

These are my notes on using some MicroPython specific tools in relation to a ESP32-DevKitC board.

There are many tutorials and youtube videos that constantly encourage users to install tools and packages into their system-level libraries. (If you need to use sudo when you pip install foo, you are installing it as a system level library.) Please, Please, Please take the time to learn the basics of virtual environment. If you are a developer/hacker/maker – save yourself lots of frustration by using virtual environments.

A virtual environment is an isolated Python environment it contains all the necessary executables to use the packages that a Python project would need. It allows you use the desired version of a specific library and isolates that library from other virtual environments and the system and user level libraries. It allows you to easily define what packages are required to reproduce your work.

A short primer based on Python3.

To Create and setup a Python3 virtualenv

# nav to where you want to create your project then ... $ python3 -m venv your-project-name $ cd your-project-name $ source bin/activate (your-project-name) $ pip install --upgrade pip

Now just use pip and skip the sudo and kludging up your system level install.

To Deactivate a virtual environment

(your-project-name) $ deactivate $

to activate an existing virtual environment

$ source bin/activate (your-project-name) $

Mandatory link to the documentation.

Now you can list the packages you have installed for THIS project with pip:

(your-project-name) $ pip freeze pkg-resources==0.0.0 (your-project-name) $ pip list pip (9.0.1) pkg-resources (0.0.0) setuptools (20.7.0)

May the Zen of Python be with you.

Categories: FLOSS Project Planets

Enrico Zini: Systemd unit files

Planet Debian - Sat, 2017-09-23 18:00

These are the notes of a training course on systemd I gave as part of my work with Truelite.

Writing .unit files

For reference, the global index with all .unit file directives is at man systemd.directives.

All unit files have a [Unit] section with documentation and dependencies. See man systemd.unit for documentation.

It is worth having a look at existing units to see what they are like. Use systemctl --all -t unittype for a list, and systemctl cat unitname to see its content wherever it is installed.

For example: systemctl cat graphical.target. Note that systemctl cat adds a line of comment at the top so one can see where the unit file is installed.

Most unit files also have an [Install] section (also documented in man systemd.unit) that controls what happens when enabling or disabling the unit.

See also:

.target units

.target units only contain [Unit] and [Install] sections, and can be used to give a name to a given set of dependencies.

For example, one could create a remote-maintenance.target unit, that when brought up activates, via dependencies, a set of services, mounts, network sockets, and so on.

See man systemd.target

See systemctl --all -t target for examples.

special units

man systemd.special has a list of units names that have a standard use associated to them.

For example, ctrl-alt-del.target is a unit that is started whenever Control+Alt+Del is pressed on the console. By default it is symlinked to reboot.target, and you can provide your own version in /etc/systemd/system/ to perform another action when Control+Alt+Del is pressed.

User units

systemd can also be used to manage services on a user session, starting them at login and stopping them at logout.

Add --user to the normal systemd commands to have them work with the current user's session instead of the general system.

See systemd/User in the Arch Wiki for a good description of what it can do.

Categories: FLOSS Project Planets

Calvin Spealman: On Pruning Your Passions [MOVED]

Planet Python - Sat, 2017-09-23 16:31
We live in a hobby-rich world. There is no shortage of pastimes to grow a passion for. There is a shortage of one thing: time to indulge those passions. If you're someone who pours your heart into that one thing that makes your life worthwhile, that's a great deal. But, what if you've got no shortage of interests that draw your attention and you realize you will never have the time for all of them?

If I look at all the things I'd love to do with my life as a rose bush I'm tending, I realize that careful pruning is essential for the best outcome. This is a hard lesson to learn, because it can mean cutting beautiful flowers and watching the petals fall to the ground to wither. It has to be done.

I have a full time job that takes a lot of my mental energy. I have a wife and a son and family time is very important in my house. I try to read more, and I want to keep up with new developments in my career, and I'm trying to make time for simple, intentional relaxing to lower my anxiety and stress. That doesn't leave a lot of room to pursue any of these hobbies.

I used to play the guitar, if only a bit.

I've always had my eye on becoming a writer.

Software development began as a passion hobby, and now that it is a carrier I still feel that draw to it outside of work.

A lot of my life was spent under the assumption that I would end up in some career as an artist, and I was even on a trajectory towards art school in my teens.

But there aren't enough days in the year, or hours in any of those days, to scratch 100% of those itches.

So, I'm committing to saying "No" to myself more often. When I'm looking for a small app or tool and can't find just the right thing, I'm going to say "No" to building my own, instead of making the best option work. When NaNoWriMo rolls around next year, I'm not going to cause myself anxiety over the "Will I? Won't I?" leading up, and I'm going to admit that it just doesn't work for me. When I end my work day, I'm going to leave the web development at work.

I will be saying "No" to myself on all these interests so I can direct my "Yes" whole heartedly to one: my blossoming foray into game development. And this is a really deliberate choice! Game development is what got me into computers and into programming. But, its also something multi-faceted in a way that few other pursuits are. By throwing myself fully into my game projects, I'll be able to spend time created art, to code outside of work and learn new techniques and paradigms, and to tell stories.

I'm putting down a lot of interests, and shelving a lot of personal projects. I have dozens of bits of code that'll only collect dust from now on, even though I think of them often and constantly feel the pull to hack on them in the evening or weekends. But, I have convinced myself this is for the best. I'm working on making 2017 a big year for me, and I can't do that when I'm pulled in a thousand directions.

Learning to give up just may be the ticket to finally succeeded.

Categories: FLOSS Project Planets

Dirk Eddelbuettel: RcppCNPy 0.2.7

Planet Debian - Sat, 2017-09-23 15:07

A new version of the RcppCNPy package arrived on CRAN yesterday.

RcppCNPy provides R with read and write access to NumPy files thanks to the cnpy library by Carl Rogers.

This version updates internals for function registration, but otherwise mostly switches the vignette over to the shiny new pinp two-page template and package.

Changes in version 0.2.7 (2017-09-22)
  • Vignette updated to Rmd and use of pinp package

  • File src/init.c added for dynamic registration

CRANberries also provides a diffstat report for the latest release. As always, feedback is welcome and the best place to start a discussion may be the GitHub issue tickets page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Categories: FLOSS Project Planets

Dirk Eddelbuettel: RcppClassic 0.9.8

Planet Debian - Sat, 2017-09-23 15:06

A bug-fix release RcppClassic 0.9.8 for the very recent 0.9.7 release which fixes a build issue on macOS introduced in 0.9.7. No other changes.

Courtesy of CRANberries, there are changes relative to the previous release.

Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Categories: FLOSS Project Planets

Ned Batchelder: Beginners and experts

Planet Python - Sat, 2017-09-23 11:13

I gave a talk at Boston Python the other night. It started as an exposition of the point matching algorithm I've previously written about on this blog. But as I thought about my side project more, I was interested to talk about the challenges I faced while building it. Not because the specifics were so interesting, but because they were typical problems that all software projects face.

And in particular, I wanted to underscore this point: software is hard, even for experts. Experts have the same struggles that beginners do.

I used this tweet as an illustration:

I love the raw emotion on the two boys' faces. They perfectly illustrate both the frustration and exhilaration of writing software.

But here's what beginners might not understand: beginners think beginners feel the frustration, and experts feel the exhilaration. As any expert will tell you, experts feel plenty of frustration. They feel like that left-hand kid a lot.

The difference between beginners and experts is that experts are familiar with that frustration. They encounter it all the time, as they deal with new unfamiliar technologies, or a thorny bug, or just when they are too tired to tackle the problem before them. They know the frustration is because they are facing a challenging problem. They know it isn't a reflection of their self-worth or abilities. Experts know the feeling will pass, and they have techniques for dealing with the frustration.

When beginners get frustrated, they can start to worry that they are not cut out for software, or they are dumb, or that everyone else gets it and they don't.

The good news for beginners is: this isn't about you. Software is difficult. We build layer upon layer of leaky abstractions, of higher and higher realms of virtualization and complexity. There's no natural limit to how high our towers of complexity can go, so we are forced to try to understand them, or tame them, and it's very hard. Our languages and tools are obscure and different and new ones are invented all the time. Frustration is inevitable.

The bad news for beginners is: this feeling won't stop. If you do your software career right, then you will always be a newb at something. Sure, you can master a specialty, and stick with it, but that can get boring. And that specialty might dwindle away leaving you stranded.

You will be learning new things forever. That feeling of frustration is you learning a new thing. Get used to it.

Categories: FLOSS Project Planets

Evennia: Evennia in pictures

Planet Python - Sat, 2017-09-23 08:44
This article describes the MU* development system Evennia using pictures!  

This article was originally written for Optional RealitiesSince it is no longer available to read on OR, I'm reposting it in full here.
Figure 1: The parts of the Evennia library

Evennia is a game development library. What you see in Figure 1 is the part you download from us. This will not run on its own, we will soon initialize the missing “jigsaw puzzle” piece on the left. But first let’s look at what we’ve got.
Looking at Figure 1 you will notice that Evennia internally has two components, the Portal and the Server. These will run as separate processes.
The Portal tracks all connections to the outside world and understands Telnet protocols, websockets, SSH and so on. It knows nothing about the database or the game state. Data sent between the Portal and the Server is protocol-agnostic, meaning the Server sends/receives the same data regardless of how the user is connected. Hiding behind the Portal also means that the Server can be completely rebooted without anyone getting disconnected.
The Server is the main “mud driver” and handles everything related to the game world and its database. It's asynchronous and uses Twisted. In the same process of the Server is also the Evennia web server component that serves the game’s website. That the Server and webserver are accessing the database in the same process allows for a consistent game state without any concerns for caching or race condition issues.
Now, let’s get a game going. We’ll call it mygame. Original, isn’t it?
Figure 2: The full setup for mygame
After installing evennia you will have the evennia command available. Using this you create a game directory - the darker grey piece in Figure 2 that was missing previously. This is where you will create your dream game!
During initialization, Evennia will create Python module templates in mygame/ and link up all configurations to make mygame a fully functioning, if empty, game, ready to start extending. Two more commands will create your database and then start the server. From this point on, mygame is up and running and you can connect to your new game with telnet on localhost:4000 or by pointing your browser to http://localhost:4001.
Now, our new mygame world needs Characters, locations, items and more! These we commonly refer to as game entities. Let’s see how Evennia handles those.
Figure 3: The Database Abstraction of Evennia entities
Evennia is fully persistent and abstracts its database in Python using Django. The database tables are few and generic, each represented by a single Python class. As seen in Figure 3, the example ObjectDB Python class represents one database table. The properties on the class are the columns (fields) of the table. Each row is an instance of the class (one entity in the game).
Among the example columns shown is the key (name) of the ObjectDB entity as well as a Foreign key-relationship for its current “location”. From the above we can see that Trigger is in the Dungeon, carrying his trusty crossbow Old Betsy!
The db_typeclass_path is an important field. This is a python-style path and tells Evennia which subclass of ObjectDB is actually representing this entity.
Figure 4: Inheriting classes to customize entities

In Figure 4 we see the (somewhat simplified) Python class inheritance tree that you as an Evennia developer will see, along with the three instanced entities.
ObjectDB represents stuff you will actually see in-game and its child classes implement all the handlers, helper code and the hook methods that Evennia makes use of. In your mygame/ folder you just import these and overload the things you want to modify. In this way, the Crossbow is modified to do the stuff only crossbows can do and CastleRoom adds whatever it is that is special about rooms in the castle.
When creating a new entity in-game, a new row will automatically be created in the database table and then “Trigger” will appear in-game! If we, in code, search the database for Trigger, we will get an instance of the Character class back - a Python object we can work with normally.
Looking at this you may think that you will be making a lot of classes for every different object in the game. Your exact layout is up to you but Evennia also offers other ways to customize each individual object, as exemplified by Figure 5.
Figure 5: Adding persistent Attributes to a game entity.

The Attribute is another class directly tied to the database behind the scenes. Each Attribute basically has a key, a value and a ForeignKey relation to another ObjectDB. An Attribute serializes Python constructs into the database, meaning you can store basically any valid Python, like the dictionary of skills seen in Figure 5. The “strength” and “skills” Attributes will henceforth be reachable directly from the Trigger object. This (and a few other resources) allow you to create individualized entities while only needing to create classes for those that really behave fundamentally different.

Figure 6: Sessions, Players and Objects

Trigger is most likely played by a human. This human connects to the game via one or more Sessions, one for each client they connect with. Their account on mygame is represented by a PlayerDB entity. The PlayerDB holds the password and other account info but has no existence in the game world. Through the PlayerDB entity, Sessions can control (“puppet”) one or more ObjectDB entities in-game.
In Figure 6, a user is connected to the game with three Sessions simultaneously. They are logged in to their Player account Richard. Through these Sessions they are simultaneously puppeting the in-game entities Trigger and Sir Hiss. Evennia can be configured to allow or disallow a range of different gaming styles like this.
Now, for users to be able to control their game entities and actually play the game, they need to be able to send Commands.
Figure 7: Commands are Python classes too

Commands represent anything a user can input actively to the game, such as the look command, get, quit, emote and so on.
Each Command handles both argument parsing and execution. Since each Command is described with a normal Python class, it means that you can implement parsing once and then just have the rest of your commands inherit the effect. In Figure 7, the DIKUCommand parent class implements parsing of all the syntax common for all DIKU-style commands so CmdLook and others won’t have to.
Figure 8: Commands are grouped together in sets and always associated with game entities.

Commands in Evennia are always joined together in Command Sets. These are containers that can hold many Command instances. A given Command class can contribute Command instances to any number of Command Sets. These sets are always associated with game entities. In Figure 8, Trigger has received a Command Set with a bunch of useful commands that he (and by extension his controlling Player) can now use.
Figure 9: Command Sets can affect those around them

Trigger’s Command Set is only available to himself. In Figure 8 we put a Command Set with three commands on the Dungeon room. The room itself has no use for commands but we configure this set to affect those inside it instead. Note that we let these be different versions of these commands (hence the different color)! We’ll explain why below.
Figure 10: The name Command “Set” is not just a name

Command Sets can be dynamically (and temporarily) merged together in a similar fashion as Set Theory, except the merge priority can be customized. In Figure 10 we see a Union-type merger where the Commands from Dungeon of the same name temporarily override the commands from Trigger. While in the Dungeon, Trigger will be using this version of those commands. When Trigger leaves, his own Command Set will be restored unharmed.
Why would we want to do this? Consider for example that the dungeon is in darkness. We can then let the Dungeon’s version of the look command only show the contents of the room if Trigger is carrying a light source. You might also not be able to easily get things in the room without light - you might even be fumbling randomly in your inventory!
Any number of Command Sets can be merged on the fly. This allows you to implement multiple overlapping states (like combat in a darkened room while intoxicated) without needing huge if statements for every possible combination. The merger is non-destructive, so you can remove cmdsets to get back previous states as needed.

… And that’s how many illustrations I have the stamina to draw at this time. Hopefully this quick illustrated dive into Evennia helps to clarify some of the basic features of the system!
Categories: FLOSS Project Planets

Evennia: Evennia 0.7 released

Planet Python - Sat, 2017-09-23 07:12
As of today Evennia 0.7 is officially out! Evennia is a Python framework and -server for creating text-based multiplayer games (MU*).

A big thank you to all collaborators that have helped with code and testing along the way!

Here is the hefty forum post that details how you migrate to Evennia 0.7.

Evennia 0.7 comes with a range of changes and updates (these are just the ones merged from the latest devel branch, a lot more has happened since 0.6 that were already in master): 
  • Evennia separates Player objects from Character objects in that the former is an OOC entity that can puppet one or more of the latter. The name Player was a source of some confusion since this is used differently in other code bases. Player has now been renamed to Account to make its function clearer.
  • Evennia's in-built website now uses our own theme and bootstrap under the hood to make it easier to modify as well as rescale it better on mobile devices (webclient is not updated at this point).
  • Shared logins between webclient and website, with the ability to log out of each independently of the other if so desired.
  • Prefix-ignoring - All default commands are now renamed without their @-prefixes. Both @examine, +examine or examine will now all point to the same command. You can customize which prefixes Evennia simply ignores when searching for a command. The mechanic is clever though - if you create a command with a specific key "+foo", then that will still work and won't clash with another command named just "foo".
  • Easy pause mechanisms using yield statements directly in Command code (just do yield 10 in your command code to have it pause ten seconds without blocking anyone else). You can also do retval = yield "Do you want to accept?" and have the command pause (non-blocking) for the player's input before continuing.
New contribs (optional plugins) (selection, new since 0.6 but many have been available on master branch for a while):
  • The optional In-Game-Python contrib by Vincent le Geoff allows for coding and scripting in-game using full-fledged Python to make events and triggers. It's not a safe softcode-style language (you have the full power of Python which is not good for untrusted users) but is intended for trusted builders to make complex scripting from the command line.
  • Wilderness/maps - Creation of dynamic or static maps based on ascii-maps for dynamically creating rooms and to show when moving around. (titeuf87, Cloud Keeper)
  • Full turn-based combat system, meant to expand for a desired system (battlejenkins)
  • Alternative Unix-style command base for Evennia, for those that prefer to enter commands like you do on the unix command line (with --flags etc) (Vincent le Geoff)
  • Multidescer, which together with Evennia's nick replacement system can be heavily customized in-game (me).
  • Mail - a @brandymail-style in-game mail-system (grungies1138)
  • Clothing system, for layered outfits adding to the wearer's desc when worn (battlejenkins).
A more technical list from the main announcement post: 
  • EvMenu formatting functions now part of class - EvMenu no longer accepts formatting functions as inputs, these are now part of the EvMenu class. To override the formatting of EvMenu nodes you should now override EvMenu and replace the format method you want. This brings EvMenu more in line with other services in Evennia, all of which are built around overriding. 
  • Scripts are now valid message senders - Also Scripts can now act as "sender" of a Msg, for example to a Channel or a player.
  • Due to the new prefix-ignoring above, @desc (builder-level description-setting) was renamed to setdesc to make it clearly separate from desc (used by a player setting their own description). This was actually the only clash we had to resolve this way in the default commands. 
  • Permission Hierarchy names change - To make Evennia's permission hierarchy better reflect how Evennia actually works, the old Players, Player Helpers, Builders, Wizards, Immortals hierarchy has changed to Player, Helper, Builder, Admin, Developer. Singular/Plural form is now ignored so Builder and Builders will both work (this was a common source of simple errors) Old permissions will be renamed as part of the migration process. The distribution of responsibilities has not changed: Wizards (which in some other systems was the highest level, which confused some) always had the power to affect player accounts (like an admin) whereas Immortals had server-level access such as @py - that is, they were developers. The new names hopefully makes this distinction clearer.  
  • All manager methods now return querysets - Some of these used to return lists which was a throwback to an older version of Typeclasses. With manager methods returning querysets one can chain queries onto their results like you can with any django query.
  • PrettyTable was removed from the codebase. You can still fetch it for your game if you prefer (it's on pypi). But EvTable has more functionality as well as color-support. 
  • Add **kwargs support to all object at_* hooks - All object at_ hooks, such as at_look, at_give etc now has an additional **kwarg argument. This allows developers to send arbitrary data to those hooks so they can expand on them without changing the API.  
  • Remove {-color tags - The use of {r, {n etc was deprecated for years and have now been completely removed from Evennia's core in favor of only one form, namely |r, |n etc. However, the color parser is now pluggable and the {-style as well as the %c style color tags etc can be re-added to your game since they are now in a contrib.  
  • Updated hooks for say/whisper commands - There are now at_before/after_say sub-hooks to allow say to be more easily customized. There is still ongoing discussion on the best way to handle this (see e.g. this PR).  
  • More compact distribution of ports - Before, Evennia distributed its ports widely, such as using 4000, 4001, 8000, 8001, 5001, 8022, some of which appeared close but actually was unrelated in functionality. They were also scattered throughout the settings file. The port definitions have now all moved more closer together, at the top of the settings file. Evennia will now use ports 4000-4006 by default (fewer depending on which protocols you want to start). This should make it easier to assign port ranges when wanting to run multiple Evennia instances on the same machine.  
  • Change how auto-login works - It used to be that once you logged into the website, every time you went to the webclient you would auto-login, even if you manually quite the webclient before. Only way to really log out (to change account, say) would be to first log out of the website. Now this will work more intuitively - you will still auto-login but if you log out of the webclient you "decouple" from the login state of the website and need to re-login to the webclient next time, even if you are still logged into the website.  
  • Better idle disconnects - This is a long-running pet peeve for many. Idle timeouts will now honor a particular lock. Setting this lock allows selected entities (for example staff and bots) to avoid getting timed out like everyone else.
  • A slew of bug fixes and other tweaks. See here for the list.
    Categories: FLOSS Project Planets

    Iain R. Learmonth: VM on bhyve not booting

    Planet Debian - Sat, 2017-09-23 05:45

    Last night I installed updates on my FreeNAS box and rebooted it. As expected my network died, but then it never came back, which I hadn’t expected.

    My FreeNAS box provides backup storage space, a local Debian mirror and a mirror of talks from recent conferences. It also runs a couple of virtual machines and one of these provides my local DNS resolver.

    I hooked up the VNC console to the virtual machine and the problem looked to be that it was booting from the Debian installer CD. I removed the CD from the VM and rebooted, thinking that would be the end of it, but nope:

    The EFI shell presented where GRUB should have been

    I put the installer CD back and booted in “Rescue Mode”. For some reason, the bootloader installation wasn’t working, so I planned to reinstall it. The autopartition layout for Debian with EFI seems to use /dev/sda2 for the root partition. When you choose this it will see that you have an EFI partition and offer to mount it for you too.

    When I went to install the bootloader, I saw another option that I didn’t know about: “Force GRUB installation in removable media path”. In the work I did on live-wrapper I had only ever dealt with this method of booting, I didn’t realise that there were other methods. The reasoning behind this option can be found in detail in Debian bug #746662. I also found mjg59’s blog post from 2011 useful in understanding this.

    Suffice is to say that this fixed the booting issue for me in this case. I haven’t investigated this much futher so I can’t be certain of any reproducable steps to this problem, but I did also stumble across this forum post which essentially gives the manual steps that are taken by that Rescue Mode option in order to fix the problem. I think the only reason I hadn’t run into this before now is that the VMs hadn’t been rebooted since their installation.

    Categories: FLOSS Project Planets

    KStars 2.8.4 aka Juli is released!

    Planet KDE - Sat, 2017-09-23 03:30
    Less than two weeks after the release of KStars 2.8.3 comes another minor bugfix release. Download KStars 2.8.4 for Windows, MacOS, and Linux.

    Major highlights:

    • Robert Barlow submitted a patch to add elevation information for all cities. The information was built from Google data. It closes Bug #382462. The elevation data is also now sent to the INDI drivers.
    • Stars and Deep Sky Objects labels are now zoom-dependent so they appear larger when zoomed in which improves usability.
    • Rotators are now fully tested and supported with Ekos capture and align module. In Ekos align module, use Load & Slew to go to any arbitrary target, and then have your rotator exactly match the orientation of the target in the image!
    • Fixed several issues with internationalization of some strings discovered by Khalid AlAjaji. Khalid also submitted significant translations for KStars in Arabic!
    • Jérôme Sonrier submitted a fresh new updated Daylight Saving rules. He is also working on getting rid of the static TZ rules in KStars and using the ones provided by the system.
    • Added PSF (Point Spread Function) convoluted star detection algorithm adapted from PHD2 to improve auto-selection of stars during guide module calibration phase.
    • Fixed bug in processing Load&Slew data when some keywords are missing.
    • Fix layout issues for RTL languages.

    This release is dedicated to Juli, my lovely German Shepard companion for the last 7 years. She is accompanied here by Tommy when he was just a small puppy back then. Long live and prosper my good girl!

    Categories: FLOSS Project Planets

    Stefan Behnel: What's new in Cython 0.27?

    Planet Python - Sat, 2017-09-23 01:15

    Cython 0.27 is freshly released and comes with several great improvements. It finally implements all major features that were added in CPython 3.6, such as async generators and variable annotations. The long list of new features and resolved issues can be found in the changelog, or in the list of resolved tickets.

    Probably the biggest new feature is the support for asynchronous generators and asynchronous comprehensions, as specified in PEP 525 and PEP 530 respectively. They allow using yield inside of async coroutines and await inside of comprehensions, so that the following becomes possible:

    async def generate_results(source): async for i in source: yield i ** 2 ... d = {i: await result for i, result in enumerate(async_results)} ... l = [s for c in more_async_results for s in await c]

    As usual, this feature is available in Cython compiled modules across all supported Python versions, starting with Python 2.6. However, using async cleanup in generators, e.g. in a finally-block, requires CPython 3.6 in order to remember which I/O-loop the generator must use. Async comprehensions do not suffer from this.

    The next big and long awaited feature is support for PEP 484 compatible typing. Both signature annotations (PEP 484) and variable annotations (PEP 526) are now parsed for Python types and cython.* types like list or cython.int. Complex types like List[int] are not currently evaluated as the semantics are less clear in the context of static compilation. This will be added in a future release.

    One special twist here is exception handling, which tries to mimic Python more closely than the defaults in Cython code. Thus, it is no longer necessary to explicitly declare an exception return value in code like this:

    @cython.cfunc def add_1(x: cython.int) -> cython.int: if x < 0: raise ValueError("...") return x + 1

    Cython will automatically return -1 (the default exception value for C integer types) if an exception is raised and check for exceptions after calling it. This is identical to the Cython signature declaration except? -1.

    In cases where annotations are not meant as static type declarations during compilation, the extraction can be disabled with the compiler directive annotation_typing=False.

    The new release brings another long awaited feature: automatic ``__richcmp__()`` generation. Previously, extension types required a major difference to Python classes with respect to the special methods for comparison, __eq__, __lt__ etc. Users had to implement their own special __richcmp__() method which implemented all comparisons at once. Now, Cython can automatically generate an efficient __richcmp__() method from the normal comparison methods, including inherited base type implementations. This brings Python classes and extension types another huge step closer.

    To bring extension modules also closer to Python modules, Cython now implements the new extension module initialisation process of PEP 489 in CPython 3.5 and later. This makes the special global names like __file__ and __path__ correctly available to module level code and improves the support for module-level relative imports. As with most internal features, existing Cython code will benefit from this by simple recompilation.

    As a last feature worth mentioning, the IPython/Jupyter magic integration gained a new option %%cython --pgo for easy profile guided optimisation. This allows the C compiler to take better decisions during its optimisation phase based on a (hopefully) realistic runtime profile. The option compiles the cell with PGO settings for the C compiler, executes it to generate the runtime profile, and then compiles it again using that profile for C compiler optimisation. This is currently only tested with gcc. Support for other compilers can easily be added to the IPythonMagic.py module and pull requests are welcome, as usual.

    By design, the Jupyter cell itself is responsible for generating a suitable profile. This can be done by implementing the functions that should be optimised via PGO, and then calling them directly in the same cell on some realistic training data like this:

    %%cython --pgo def critical_function(data): for item in data: ... # execute function several times to build profile from somewhere import some_typical_data for _ in range(100): critical_function(some_typical_data)

    Together with the improved module initialisation in Python 3.5 and later, you can now also distinguish between the profile and non-profile runs as follows:

    if "_pgo_" in __name__: ... # execute critical code here
    Categories: FLOSS Project Planets

    Russell Coker: Converting Mbox to Maildir

    Planet Debian - Fri, 2017-09-22 23:52

    MBox is the original and ancient format for storing mail on Unix systems, it consists of a single file per user under /var/spool/mail that has messages concatenated. Obviously performance is very poor when deleting messages from a large mail store as the entire file has to be rewritten. Maildir was invented for Qmail by Dan Bernstein and has a single message per file giving fast deletes among other performance benefits. An ongoing issue over the last 20 years has been converting Mbox systems to Maildir. The various ways of getting IMAP to work with Mbox only made this more complex.

    The Dovecot Wiki has a good page about converting Mbox to Maildir [1]. If you want to keep the same message UIDs and the same path separation characters then it will be a complex task. But if you just want to copy a small number of Mbox accounts to an existing server then it’s a bit simpler.

    Dovecot has a mb2md.pl script to convert folders [2].

    cd /var/spool/mail mkdir -p /mailstore/example.com for U in * ; do ~/mb2md.pl -s $(pwd)/$U -d /mailstore/example.com/$U done

    To convert the inboxes shell code like the above is needed. If the users don’t have IMAP folders (EG they are just POP users or use local Unix MUAs) then that’s all you need to do.

    cd /home for DIR in */mail ; do U=$(echo $DIR| cut -f1 -d/) cd /home/$DIR for FOLDER in * ; do ~/mb2md.pl -s $(pwd)/$FOLDER -d /mailstore/example.com/$U/.$FOLDER done cp .subscriptions /mailstore/example.com/$U/ subscriptions done

    Some shell code like the above will convert the IMAP folders to Maildir format. The end result is that the users will have to download all the mail again as their MUA will think that every message had been deleted and replaced. But as all servers with significant amounts of mail or important mail were probably converted to Maildir a decade ago this shouldn’t be a problem.

    Related posts:

    1. Why Cyrus Sucks I’m in the middle of migrating a mail server away...
    2. Moving a Mail Server Nowadays it seems that most serious mail servers (IE mail...
    3. Mail Server Training Today I ran a hands-on training session on configuring a...
    Categories: FLOSS Project Planets

    Glyph Lefkowitz: Photo Flow

    Planet Python - Fri, 2017-09-22 21:27

    Hello, the Internet. If you don’t mind, I’d like to ask you a question about photographs.

    My spouse and I both take pictures. We both anticipate taking more pictures in the near future. No reason, just a total coincidence.

    We both have iPhones, and we both have medium-nice cameras that are still nicer than iPhones. We would both like to curate and touch up these photos and actually do something with them; ideally we would do this curation collaboratively, whenever either of us has time.

    This means that there are three things we want to preserve:

    1. The raw, untouched photographs, in their original resolution,
    2. The edits that have been made to them, and
    3. The “workflow” categorization that has been done to them (minimally, “this photo has not been looked at”, “this photo has been looked at and it’s not good enough to bother sharing”, “this photo has been looked at and it’s good enough to be shared if it’s touched up”, and “this has been/should be shared in its current state”). Generally speaking this is a “which album is it in” categorization.

    I like Photos. I have a huge photo library with years of various annotations in it, including faces (the only tool I know of that lets you do offline facial recognition so you can automatically identify pictures of your relatives without giving the police state a database to do the same thing).

    However, iCloud Photo Sharing has a pretty major issue; it downscales photographs to “up to 2048 pixels on the long edge”, which is far smaller even than the 12 megapixels that the iPhone 7 sports; more importantly it’s lower resolution than our television, so the image degradation is visible. This is fine for sharing a pic or two on a small phone screen, but not good for a long-term archival solution.

    To complicate matters, we also already have an enormous pile of disks in a home server that I have put way too much energy into making robust; a similarly-sized volume of storage would cost about $1300 a year with iCloud (and would not fit onto one account, anyway). I’m not totally averse to paying for some service if it’s turnkey, but something that uses our existing pile of storage would definitely get bonus points.

    Right now, my plan is to dump all of our photos into a shared photo library on a network drive, only ever edit them at home, try to communicate carefully about when one or the other of us is editing it so we don’t run into weird filesystem concurrency issues, and hope for the best. This does not strike me as a maximally robust solution. Among other problems, it means the library isn’t accessible to our mobile devices. But I can’t think of anything better.

    Can you? Email me. If I get a really great answer I’ll post it in a followup.

    Categories: FLOSS Project Planets

    Justin Mason: Links for 2017-09-22

    Planet Apache - Fri, 2017-09-22 19:58
    Categories: FLOSS Project Planets

    Enrico Zini: Systemd on the command line

    Planet Debian - Fri, 2017-09-22 18:00

    These are the notes of a training course on systemd I gave as part of my work with Truelite.

    Exploring the state of a system
    • systemctl status [unitname [unitname..]] show status of one or more units, or of the whole system. Glob patterns also work: systemctl status "systemd-fsck@*"
    • systemctl list-units or just systemctl show a table with all units, their status and their description
    • systemctl list-sockets lists listening sockets managed by systemd and what they activate
    • systemctl list-timers lists timer-activated units, with information about when they last ran and when they will run again
    • systemctl is-active [pattern] checks if one or more units are in active state
    • systemctl is-enabled [pattern] checks if one or more units are enabled
    • systemctl is-failed [pattern] checks if one or more units are in failed state
    • systemctl list-dependencies [unitname] lists the dependencies of a unit, or a system-wide dependency tree
    • systemctl is-system-running check if the system is running correctly, or if some unit is in a failed state
    • systemd-cgtop like top but processes are aggregated by unit
    • systemd-analyze produces reports on boot time, per-unit boot time charts, dependency graphs, and more
    Start and stop services

    Similar to the System V service command, systemctl provides commands to start/stop/restart/reload units or services:

    • start: starts a unit if it is not already started
    • stop: stops a unit
    • restart: starts or restarts a unit
    • reload: tell a unit to reload its configuration (if it supports it)
    • try-restart: restarts a unit only if it is already active, otherwise do nothing, to prevent accidentally starting a service
    • reload-or-restart: tell a unit to reload its configuration if supported, otherwise restart it
    • try-reload-or-restart: tell a unit to reload its configuration if supported, otherwise restart it. If the unit is not already active, do nothing to prevent accidentally starting a service.
    Changing global system state

    systemctl has halt, poweroff, reboot, suspend, hibernate, and hybrid-sleep commands to tell systemd to reboot, power off, suspend and so on. kexec and switch-root also work.

    The rescue and emergency commands switch the system to rescue and emergency mode (see man systemd.special. systemctl default switches to the default mode, which also happens when exiting the rescue or emergency shell.

    Run services at boot

    systemd does not implement runlevels, and services start at boot based on their dependencies.

    To start a service at boot, you add to its .service file a WantedBy= dependency on a well-known .target unit.

    At boot, systemd brings up the whole chain of dependency started from a default unit, and that will eventually activate also your service.

    See systemctl get-default for what unit is currently the default in your system. You can change it via the systemd.unit= kernel command line, so you can configure multiple entries in the boot loader that boot the system running different services. For example systemd.unit=rescue.target for a rescue mode, systemd.unit=multi-user.target for a non-graphical mode, or add your own .target file to implement new system modes.

    See systemctl list-units -t target --all for a list of all currently available targets in your system.

    • systemctl enable unitname enables the unit to start at boot, by creating symlinks to it in the .wants directory of the units listed in its WantedBy= configuration
    • systemctl disable unitname removes the symlinks created by enable
    • systemctl reenable unitname removes and readds the symlinks for when you changed WantedBy=

    Notes: systemctl start activates a unit right now, but does not automatically enable it at boot systemctl enable enables a unit at boot, but does not automatically start it right now * a disabled unit can still be activated if another unit depends on it

    To disable a unit so that it will never get started even if another unit depends on it, use systemctl mask unitname. Use systemctl unmask unitname to undo the masking.

    Reloading / restarting systemd

    systemctl daemon-reload tells systemd to reload its configuration.

    systemctl daemon-reexec tells systemd to restart iself.

    Categories: FLOSS Project Planets
    Syndicate content