2020-12-25

Devils: Massimo Ruggiero's apartment building

If you enjoyed watching Devils, like me, and if you tend to obsess over tiny details that have absolutely nothing to do with the story being told, like me, and you lived in that general area, like me, then you may be curious of the location of Massimo's apartment.

In several shots we can see views of both The Gherkin and the north side of Canary Wharf, so that means the building is in and around E1/E2/E3 zip codes. But London is huge, and while yes i tried to just browse Google Maps for clues, that didn't produce any result.

The final clue is revealed in the 9th episode, at minute 20:10, and precisely with this shot:

at the bottom left we can see the entrance sign for Tobacco Dock, located at:

So right there, just off to the right from the entrance, it's the apartment building: Park Vista Tower.

The exact apartment number, i don't know, it's probably somewhere on the door or on the door frame, which can probably be seen one of the times when Sofia or others come to visit Massimo, but since that happened mid 9th episode, i'm not gonna go back to find that clue 😀

2020-12-19

Gmail and tracker.debian.org: add a label to team emails

Recently the Python Team started the migration from Alioth mailing list to a tracker.debian.org team.

Alioth mailing list, being a real ml, included a List-Id header, which is handled by Gmail natively and which you can use to create a filter based on it.

Tracker.d.o instead uses a custom header, X-Distro-Tracker-Team, and Gmail doesn't allow to create filters on custom headers. But there's a solution: Google Apps Scripts.

1. Apps script and its configuration

To create a new apps script, click "New project" from the scripts dashboard: this will open a code editor, so now cut and paste this code (change the label name to match the one you want to use, and the header too if you're going to use this for another team):

function tagPythonTeamEmails() {
  
  // search for all recent threads (hours is the smallest interval searchable)
  var threads = GmailApp.search("newer_than:1h");
  // this is the label i want to apply to the messages
  var label = GmailApp.getUserLabelByName("Debian_Python_Pkgs");  // <-- EDIT THIS

  for (var i = 0; i < threads.length; i++) {
     
    // get all messages in a given thread
    var messages = threads[i].getMessages();

    // for each message in the thread
    for (var j = 0; j < messages.length; j++) {
      var message = messages[j];
      // needed to have access to the full email: headers + body
      var email = message.getRawContent();
      // this is the header i want to check for: if present, add label and archive
      if (email.indexOf("X-Distro-Tracker-Team: python") > -1) {  // <-- EDIT THIS
        threads[i].addLabel(label);
        threads[i].moveToArchive();
      }
    }
  }
}

(This code is largely inspired by this StackOverflow answer.)

Given we're going to perform changes to the emails, we need to add extra permissions to the script; first we need to toggle the option "View > Show manifest file" which will show the appsscript.json file. Once done, edit it and update the scope of the script to include:

  "oauthScopes": [
    ...
    "https://www.googleapis.com/auth/gmail.readonly",
    "https://www.googleapis.com/auth/gmail.modify"
  ],

These new permission will be granted when we deploy the script, which we can actually do right now: from the editor window click on "Publish > Deploy from manifest..." and then "Install add-on"; it will make the app script available to operate on Gmail by opening the usual Google grants authorization popup.

2. Schedule the app script execution

Unlike filters defined on the web UI, apps scripts are not triggered every time a mail is received (there's a pub-sub mechanism, but it doesn't look like it would work for this), so if we want to execute periodically the new script, we need to create a time-driven trigger.

The simplest way is probably from the script editor (but you can also follow this doc): "View > Current project's triggers" and then "Add trigger" from there. For my specific use-case i set:

  • event source = time-driven
  • type of time based trigger = minutes timer
  • minute interval = 5 minutes
  • Failure notification settings = notify me immediately
it's not great that you have to wait 5 minutes for getting proper labels, and while it's true you can schedule the trigger for every minute, there are further considerations for that, discussed below.

3. Additional information

Be aware of quotas and limitations for Apps scripts; in particular the "Triggers total runtime" set to "90 min / day" is what led me to set the trigger every 510 mins.

Use the scripts dashboard (doc) to monitor the script execution logs, in the "My Executions" section, and specifically the "Duration" column: the script execution time depends exclusively on the number of emails to process at every execution, so fine-tune the trigger time to stay within your quota.

The newer_than search operator doesn't allow a granularity smaller than hours (although that's not even documented), so that means we're going to scan multiple times the same emails.

UPDATE (2020-12-26): I've already had to bump from every 5 to every 10 minutes, as otherwise i'd get Exception: Service invoked too many times for one day: gmail. (remember the quota!)

2020-12-08

Python: sent emails with embedded images

to send emails with images you need to use MIMEMultipart, but the basic approach:

import smtplib

from email.mime.multipart import MIMEMultipart
from email.mime.image import MIMEImage

msg = MIMEMultipart('alternative')
msg['Subject'] = "subject"
msg['From'] = from_addr
msg['To'] = to_addr

part = MIMEImage(open('/path/to/image', 'rb').read())

s = smtplib.SMTP('localhost')
s.sendmail(from_addr, to_addr, msg.as_string())
s.quit()

will produce an email with empty body and the image as an attachment.

The better way, ie to have the image as part of the body of the email, requires to write an HTML body that refers to that image:

import smtplib

from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from email.mime.image import MIMEImage

msg = MIMEMultipart('alternative')
msg['Subject'] = "subject
msg['From'] = from_addr
msg['To'] = to_addr

text = MIMEText('<img src="cid:image1">', 'html')
msg.attach(text)

image = MIMEImage(open('/path/to/image', 'rb').read())

# Define the image's ID as referenced in the HTML body above
image.add_header('Content-ID', '<image1>')
msg.attach(image)

s = smtplib.SMTP('localhost')
s.sendmail(from_addr, to_addr, msg.as_string())
s.quit()

The trick is to define an image with a specific Content-ID and make that the only item in an HTML body: now you have an email with contains that specific image as the only content of the body, embedded in it.

Bonus point: if you want to take a snapshot of a webpage (which is kinda the reason i needed the code above) i found it extremely useful to use the Google PageSpeed Insights API; a good description on how to use that API with Python is available at this StackOverflow answer.

UPDATE (2020-12-26): I was made aware via email that some mail providers may not display images inline when the Content-ID value is too short (say, for example, Content-ID: 1). A solution that seems to work on most of the providers is using a sequence of random chars prefixed with a dot and suffixed with a valid mail domain.

2020-11-05

QNAP firmware 4.5.1.1465: disable ssh management menu

as a good boy i just upgraded my QNAP NAS to the latest available firmware, 4.5.1.1465, but after the reboot there's an ugly surprise awaiting for me

once i ssh'd into the box to do my stuff, instead of a familiar bash prompt i'm greeted by a management menu that allows me to perform some basic management tasks or quit it and go back to the shell. i dont really need this menu (in particular because i have automations that regularly ssh into the box and they are not meant to be interactive).

to disable it: edit /etc/profile and comment the line "[[ "admin" = "$USER" ]] && /sbin/qts-console-mgmt -f" (you can judge me later for sshing as root)

UPDATE (2020-12-26): as pointed out in the comments, and discovered now after an upgrade, the menu comes back up after because /etc/profile is a system file for QNAP so gets replaced/updated by them often

2020-10-23

Multiple git configurations depending on the repository path

For my work on Debian, i want to use my debian.org email address, while for my personal projects i want to use my gmail.com address.

One way to change the user.email git config value is to git config --local in every repo, but that's tedious, error-prone and doesn't scale very well with many repositories (and the chances to forget to set the right one on a new repo are ~100%).

The solution is to use the git-config ability to include extra configuration files, based on the repo path, by using includeIf:

Content of ~/.gitconfig:

[user]
    name = Sandro Tosi
    email = <personal.address>@gmail.com

[includeIf "gitdir:~/deb/"]
    path = ~/.gitconfig-deb

Every time the git path is in ~/deb/ (which is where i have all Debian repos) the file ~/.gitconfig-deb will be included; its content:

[user]
    email = morph@debian.org

That results in my personal address being used on all repos not part of Debian, where i use my Debian email address. This approach can be extended to every other git configuration values.

2020-06-11

Installing prometheus snmp_exporter on a QNAP nas

I've this project in the background about creating a grafana dashboard for a QNAP nas.

As the summer is ramping up, i wanted to figure out the temperature of the nas throughout the day, so.. prometheus + grafana to the rescue!

I wrote the instructions to install snmp_exporter on a linux-based QNAP nas. For now i'm using an already existing dashboard, but it's the first step to create my own.

2020-05-08

It's a waiting game... but just how long we gotta wait?

While waiting for my priority date to become current, and with enough "quarantine time" on my hand, i just come up with a very simple Python tool to parse the USCIS Visa Bulletin to gather some data from that.

You can find code and images in this GitHub repo.

For now it only contains a single plot for the EB3 final action date; it answers a simple question: how many months ago your priority date should be if you want to file your AOS on that month. We started from FY2016, to cover the final full year of the Obama administration.

If you're interested in more classes/visas, let me know and the tool could be easily extended to cover that too. PRs are always welcome.