Dreami's Blog

Thoughts and notes of a geek

This blog is moving — June 29, 2017

This blog is moving

Hi! I’ts been almost 4 years now on this WordPress blog and it’s time to move now. I’m going for hugo, a static site generator, and since all the links break anyway, I chose to move to a new domain.

You can find my new blog at https://blog.lroellin.ch/. For my followers, there’s an RSS option, but you won’t receive anything by email or twitter.

I hope you still read my blog after moving šŸ™‚ thanks!

Hotel TV with only Component? They’re AV (Composite) compatibleĀ  — May 26, 2017
Python for Java Programmers – Part 1, Getter and Setter — May 5, 2017

Python for Java Programmers – Part 1, Getter and Setter

I’m currently in the process of learning Python. What I know is Java and C# and some script languages like PowerShell. I want to tell about my journey a bit since you’re getting in a kind of culture shock when going from Java to Python

In Java, the compiler does a great job at keeping you from doing stupid things. In Python, that’s up to you since the script is loaded as needed (not exactly in the technical meaning) and run from top to bottom (again, not exactly technically). And because all that happens quite fast and you don’t spend much time on it, the compiler won’t check such stuff for you before this code is executed.

I found a nice quote about this from The Benevolent Dictator: “We’re all consenting adults”. So it’s your job to keep an eye on what you access and if that’s possible. However keep in mind that IDEs do a great job of telling you these things, but it’s not the default for a Python instalation.

Now for the first thing we’re gonna look at: getter and setter. This will be me paraphrasing this video.

In Java, again, you write a class and put in private attributes. You then write a constructor and public getter and setter for that (ok, your IDE does). This is to stop you from breaking the encapsulation. And if you want to validate something on the setter, then you have a single place to change.

However, how many times did you actually change the setter? For me, coming from C# (which sorta has what I’m gonna show you for Python, Properties), getter and setter are noise in a class. I skip them while reading your class and that means I will skip your validation scheme on one important attribute.

Now the alternative would be to set it all public and when the time comes, change it to a private attribute and a public getter and setter. But now you have to change all client code (code that calls your class) and that can turn into a nightmare, especially if you’re developing a library and now every single piece of code using the updated version has to change. There sure are better methods, tools or frameworks for this, but I currently don’t know any.

In Python, the story is a bit different. You don’t actually declare your attributes, since (what I’ve grasped so far), definition is declaration in Python. So what you’ll do is initialize your parameters in the constructor just so you have a name you can all agree on. Then you’ll just access those attributes from the outside.

Now what if you have to validate your attribute on a later version: well, Python has you covered there. You simply define your attribute as a property, define getter and setter for that, and store the actual attribute as _attribute. The underscore is the convention that others should not touch your attribute. Remember, we’re all consenting adults. Then, no client code needs to be touched (except of course when it now breaks your validation but it’s no syntax error or the like.

How do you do that? Let me close this post with just a bit of code coming straight from StackOverflow

Before

BigX C(object):
    def __init__(self):
        self.x = None

# Getting and setting
 o = BigX
 o.x
 o.x = 5

After:

class BigX(object):
    def __init__(self):
        self._x = None
    @property
    def x(self):
        """I'm the 'x' property."""
        print "getter of x called"
        return self._x
    @x.setter
    def x(self, value):
        print "setter of x called"
        self._x = value

# Getting and setting
o = BigX
o.x # "getter of x called"
o.x = 5 # "setter of x called"
Spring/Hibernate @Transactional pitfall — April 29, 2017

Spring/Hibernate @Transactional pitfall

We’re currently doing a student project which I hope to show off in the future. However, yesterday I ran into a problem where I want to document the fix.

We have a repository method that is marked as @Transactional to delete a certain row. At first, we called that in a loop, from a method from one of our @Service classes and it worked fine. 

We then refactored it and added a method to the repository, which accepts a list and then does the looping for us, calling the other repository method in the loop. Then stuff broke. 

We got a lot of the following messages:

org.springframework.dao.InvalidDataAccessApiUsageException: No EntityManager with actual transaction available for current thread – cannot reliably process ‘remove’ call
The part “No EntityManager with actual transaction available for current thread” points to a transaction problem, which happens when you don’t use @Transactional. But we did. So what happens? 

Remember that annotations are worked out by using reflection. And reflection is, sometimes, a bit magic. And here it failed. 

The problem was, that the @Transactional on the actual remove method wasn’t picked up, when it was called from another method in the repository. So no transaction was started. 

To fix this, you have to add @Transactional to the method you call from outside your class – the one that accepted the list. That fixed the problem! 

Changing your e-mail address is (apparently) hard — March 15, 2017

Changing your e-mail address is (apparently) hard

I recently changed my email address from a @somelocalproviderinswitzerland.ch to a @gmail.com address. Mainly because I’m using Google’s software anyway and because I can enjoy a much better email client online then. That meant, obviously, changing it on a lot of sites.

Fortunately, I’m using a password manager, so I at least had a listing of accounts I’m still using. Most of the time, it went like this:

  1. Log into the website
  2. Change the email address
  3. Confirm the new address
  4. Done

I also took the opportunity and added +domain to the address I registered, so if I reveice spam I at least know where I got it from. Apparently, some sites don’t like that and I had to enter just the normal email address. That’s because a regex for email is hard.

Some other accounts are quite deeply embedded into your actual operating system. I didn’t try changing my Microsoft or my Apple account, nor my PSN account since I feared of geting logged out and losing my purchases. Maybe it works, but I don’t have much trust in it.

On other occassions, I simply could not change my address. On one site I got into a weird account state where it wouldn’t let me confirm my address (it’s already confirmed) but I couldn’t log in too (you need to confirm it). So it seems that that part of the site is not as thoroughly tested…

Some other sites apparently use mail distributors for things like newsletters. This is ok, but it took a while until those distributors got my new address or it hasn’t happened yet aswell. So updating an address needs to go through everywhere.

Some sites just don’t let you change your address. I think they’ve used it as an ID in some database so they can’t change it. The only way here is to create a new account.

You also need to know that you won’t get rid of your old address soon. It’s still around in my GitHub account and will be for years, linking commits signed with my old address to my account. These are hardcoded and can’t be changed.

What I propose (even though I don’t think this will be implemented in the future): a standard interface for changing an e-mail address (and your password while you’re at it). On every site, I spent at least some 10 seconds to 2 minutes to actually finding the screen where I could change it.

 

 

Follow-up on a sent email: Reply to All — March 1, 2017

Follow-up on a sent email: Reply to All

I think you’ve all been in this situation: you send an email and after some time, you want to follow up. Maybe you didn’t receive a reply, maybe you forgot something. So how do you send an email with the exact same receiver(s) easily, probably with the same subject?  

It turns out, that for some reason, Reply to All does exactly that. I actually got no logical explanation for that, since the normal Reply just composes a message back to you. 

If anyone knows the reason for that behavior, I’m really interested! 

Google Rewards makes your answers biasedĀ  — February 25, 2017

Google Rewards makes your answers biasedĀ 

I’m using Google Rewards on my phone. It’s an app where you can fill out surveys and get Google Play credits. It’s all fine and made by Google. 

The weird thing about it, is how much you get. Usually, it’s about 0.35 CHF which is not much, but with enough surveys you can buy small games or even a book. 

Now the problem is, you don’t always earn the same and it’s based on your answers. Typically, it goes like this:

  • To which of these shops (X, Y and Z) have you been lately? 
  • When was the last time you were in shop X? 
  • How was your impression in shop X? 

You can also select None on the first page, but guess what – you’ll only get 0.10 CHF for that and the survey is over. 

I think this wouldn’t be that much of a problem if there were 10 each day, but it’s more like 1-2 each week. So you can either earn 0.20 or 0.70 CHF each week – guess what people do after some time. 

My idea: smooth this out a bit. Right now the ratio is 1:3.5, it should go towards 1:2. That way, your answers get less biased. 

Weird HA issue on Fortinet — February 24, 2017

Weird HA issue on Fortinet

We recently had a very weird issue considering HA config sync and failover. The failover was not working when we set the slave’s priority higher to the master’s and we found configuration sync issues. This was all done on FortiOS 5.2, so I can’t really say anything about any later versions, but I think this still applies.

By the way, all the commands here are in my little cheat sheet aswell. You can find that on a GitHub repo I have set up for cheatsheets like this.

We could see that the config was not the same on both cluster members. You can see this with a simple command: diag sys ha cluster-csum. This shows the configuration checksums. This showed the following (taken from the Fortinet page about config sync issues):

================== FG100D3G13xxxxxx =================
Ā is_manage_master()=0, is_root_master()=0

debugzone
global: 89 f2 f0 0b e8 eb 0d ee f8 55 8b 47 27 7a 27 1e
root: cf 85 55 fe a7 e5 7c 6f a6 88 e5 a9 ea 26 e6 92
all: f4 62 b2 ce 81 9a c9 04 8f 67 07 ec a7 44 60 1f

checksum
global: 89 f2 f0 0b e8 eb 0d ee f8 55 8b 47 27 7a 27 1e
root: cf 85 55 fe a7 e5 7c 6f a6 88 e5 a9 ea 26 e6 92
all: f4 62 b2 ce 81 9a c9 04 8f 67 07 ec a7 44 60 1f

================== FG100D3G12xxxxxx ==================
is_manage_master()=1, is_root_master()=1

debugzone
global: 89 f2 f0 0b e8 eb 0d ee f8 55 8b 47 27 7a 27 1e
root: d8 f5 57 46 f0 b8 45 1e 00 be 45 92 a2 07 14 90
all: a7 8d cc c7 32 b5 81 a2 55 49 52 21 57 f9 3c 3b

checksum
global: 89 f2 f0 0b e8 eb 0d ee f8 55 8b 47 27 7a 27 1e
root: d8 f5 57 46 f0 b8 45 1e 00 be 45 92 a2 07 14 90
all: a7 8d cc c7 32 b5 81 a2 55 49 52 21 57 f9 3c 3b

As you can see, the cluster members have a difference in the root VDOM and therefore also in the all view. As you can see on that page, you can drill down the various levels with diagnose system ha showsum LEVEL. However, as far as I remember, there were no differences. A colleague then diff-ed the whole config and we found no differences – except in the config system ha part. As you know, Fortinet does not sync everything since there obviously are parts that don’t have the same settings on both cluster members. In that part, this is also the case since values like the priority should obviously be different.

However, it seems that while that whole part is not synced, some settings still go into that checksum calculation!

We then changed another parameter, set override enable on the slave (this was already set on the master). This basically changes the Master election algorithm to favor Priority before Uptime. So with this setting, a higher priority wins, without that, a higher uptime wins. In easy terms, it allows overriding the Master election via the priority.

After setting that configuration, we set the priority of the slave higher than the master and we had an immediate failover (like we wanted). We then checked the checksums again, and everything was fine! Setting the slave’s priority back to lower than the master’s triggered another wanted failover.

What we found out: some config parts are not synced, but they are still part of the configuration checksum. Keep that in mind when having similar issues.

UPDATE: we checked it in the lab and apparently, override is not part of the checksum calculation. So we can’t really pin it down to what actually fixed it, but it’s now ok.

How to take notes as a CS student – revised —

How to take notes as a CS student – revised

This is a follow-up to my older postĀ How to take notes as a CS student. Not everything in there is still valid.

First, I switched to a Mac starting this semester. The reason being that I need a Unix-like system but Linux is sometimes just too cumbersome. I also like that for certain tools, there’s just a “Mac” section instead of one for every kind of package manager out there.

The problem started when I saw, that OneNote is really not up to the job on the Mac. My usually 50 pages (read: images) long lectures and notes along them, are just laggy to load and scroll. It’s really not an issue on my other notebook which has about the same CPU specs as my MacBook Pro (I bought the old one for reasons).

What I now do is the following

For every lecture we have, I copy it from my sync folder to a special lecture notesĀ folder. I then open it in Preview and annotate it with notes from the lecture.

After the lecture, there’s a step I didn’t do before. In the evening or on the weekend, I go through the pages and write a summary for the current lecture. I think I’ll still have to go through the actual lectures in my study period before the exams, but it’s more a step of rethinking and maybe following up on things I didn’t have time during the lecture. You can find these atĀ GitHub. To actually find them useful, you should be able to read German, but I think you’ll see the way it works.

To actually type up these summaries, I use Typora. Some reasons I particularly chose this:

In Typora, you type in WYSIWYG but it’s still Markdown and you see just a live rendering. This helps me spot errors as I type. It does support Markdown syntax (so you can also type an asterisk and it’ll create a list), but also menu items/keyboard shortcuts for choosing your formatting as you would in a word processor. It also supports Github Flavored Markdown, a flavor I personally see as the de-facto standard.

The feature that’s made it stand out the most, is picture pasting from the clipboard. See, some graphics are best simply copied from the lecture. Typora supports this very easily. You specify a folder where Typora can copy the picture files to (should be a folder in the same folder as your Markdown file). Whenever you paste an image, Typora will save the image in that folder and automatically add that image to your Markdown.

Additionally, it supports LaTeX Math Mode inline and block-style. This is not GHF, but as it’s enabled by $$/$$$$ (as in LaTeX), anyone rendering it on GitHub will just see what looks like LaTeX and can then figure out what that means. I think this is reasonable, as LaTeX is also the de-facto standard for typing up math. And anyone not fluent in LaTeX will still see the intention and meaning of a \frac{x}{y}. Why not switch to full-blown LaTeX you ask? I tried that and have typed up summaries in LaTeX but it’s just too much friction and that’s what you gotta keep down when learning a new habit.

Also, the live rendering switches to Markdown on certain parts, like inline code rendering, when you set the cursor to it. And it’s quite intelligent, so it doesn’t do this on list asterisks. And of course, you can always switch to Source Mode and fix those pesky errors you get when importing weirdly formatted stuff.

Is this the ultimate solution? Probably not. But as last time, it works for now!

Fixing a real git history problem — February 15, 2017

Fixing a real git history problem

I’m recently involved in translating the old German comments in the LibreOffice codebase to English ones. It’s quite a challenge, since you need your translation brain and also your coder brain. The end result should be a comment that is technically correct while still sounding English. 

I ran into a small problem I made myself: at first, I used the web editor when I started, a week ago. I didn’t do any work since then until today, where I picked up again. Of course, someone else has made changes to that file, although in a different area. 

Since I didn’t like the editor, I cloned the whole repo and used the setup scripts to commit and publish from my local machine. 

Now instead of downloading the diff from the web version, I downloaded the whole file as-is and replaced the git version with that one. I then went on with my translations, eventually publishing it (applying for code review). 

Now you might already know what I did wrong. When you commit in git, you essentially create a linked list of diff patches, all pointing to each other. So when I just downloaded and re-applied my complete version, my patch included reversing the change. Oops. 

Of course, a code reviewer saw this, so nothing really happened. Now I still needed to fix this. My plan was:

  • Put my complete new file in a new branch based on master
  • Get a diff from that change someone else made
  • Apply that diff
  • Publish the new version where the patch is just my changes 

And that worked out. To get a diff for a specific commit and only the part for a specific file:

git format-patch -1 <sha> specific/file.cxx

 

Then you just apply the patch via:

 patch < file.patch

 

Note that this may not work completely, so check for the error messages patch provides. For me, I had to manually apply a small edit where it couldn’t find the surroundings. Now, if this were multiple commits, you would just apply each patch sequentially, in the right order. 

And then you can safely add, commit and publish as usual.