Solving spam problems without using captcha

Some of you know that automated spam bots for the web can be pain in the ass.

And while there are many ways to deal with this problem, most of them complicate things for the regular user and are not hacky enough to satisfy me. 🙂 For example captchas, math problem solving, etc. mostly work fine but they also scare away some of the users. There are also some paid tools (Akismet?) that helps dealing with spam but it doesn’t work on all spam.

I will share information on how I stop almost all of the automated spam messages to my WordPress blog without complicating things for the user.

 

Knowing limitations of HTTP protocol libraries and “exploiting” them

I have been writing a lot of scripts that automate GET’ing and POST’ing things to the web. There are many libraries that help doing it and one of the most popular ones (if not the most popular) is Curl.

While Curl is great, it still lacks some features, like building a zero length multipart POST data file upload part. Basically it can’t simulate file upload field, that has no file selected for uploading. And since many spam bots use Curl and similar libraries, this can be used to identify real browser and a script.

Example usage in PHP:

if(!isset($_FILES['spamcheck']) or $_FILES['spamcheck']['name']!='' or $_FILES['spamcheck']['size']!=0 )

Not just Curl

I haven’t used Curl for a year now because I switched to Perl’s LWP. And it has the same problem. I solved it by making my own function for building the multipart form data and it can simulate the behavior but it doesn’t happen by default.

And while in case of LWP it was rather simple, doing it in Curl (used in PHP) would probably require recompiling the Curl library and then recompiling the module for the programming language that is using that library.

 

Changing field names

Another trick I use is changing field names and adding additional ones that are meant to be left empty.

This method also seems to catch part of spam messages.

 

From the user point of view

User obviously doesn’t see those extra fields because they get hidden using CSS. The comment form looks like any other standard comment form.

 2013-04-12-125208_1440x900_scrot

Conclusion

I have been using these techniques for about a year now and haven’t had any spam problems since then. Also I have a huge log file with spam that was caught using these methods. This really does work. 🙂

Of course it won’t help in case of directly targeted spam but in that case captchas won’t help either.

And I understand that writing about this will probably contribute to fixing Curl and other libraries and eventually making this protection method useless. Well, at least they will finally fix those libraries! 😀

Par tiem dzeltenajiem taloniem…

Ik pa laikam saņemu jautājumus par E-taloniem kā NDEF tagiem, tāpēc izdomāju par to iedrukāt blogā.

Pēc Mifare Ultralight specifikācijas sanāk, ka izlietotu dzelteno E-talonu nevar lietot kā NFC NDEF tagu, jo pēc izlietošanas OTP (page 3) baiti nav uzstādīti ne uz 00 00 00 00 (tukšam tagam), ne  E1 10 06 00 (formatētam NDEF tagam), kā arī lock baiti (page 2, byte 2 un 3) dažās versijās (vecākām kartēm) nav uzstādīti uz 00 00. Šī iemesla dēļ, taga formatēšana beigsies ar kļūdas paziņojumu.

Tas gan nenozīmē, ka dzeltenajos E-talonos nevar rakstīt savu informāciju.

Vēl nedaudz pārsteidza izmaiņas jaunākajos E-talonos. Narvesenā pirktajiem dzeltenajiem taloniem lock baiti (page 2, byte 2 un 3) vairs netiek uzstādīti uz 70 00, kas nozīmē, ka visas lapas (4-15 ir brīvi rakstāmas).

New version of Tespeed (Terminal network speed test)

Most advanced computer users know about speedtest.net that helps testing your internet bandwidth. While it is a great tool, it has only flash version available on their web site and there are Android and iOS apps for your mobile phones, but there were no tools available for Linux servers without GUI.

Back in 2009 I made a PHP script that tested network speed, wrote the results to text file and generated PNG graphs to be shown on web. It used speedtest.net partner servers and it got the data usage from ifconfig thus making it read the total bandwidth, not just what had been left for the php script itself. I set it up on my home router so I could prove my ISP that their internet is far form what they advertised.

I also uploaded it to Sourceforge and forgot about it’s existence for some time.

Years have passed and it has been downloaded almost 17 000 times, even though it doesn’t really work in every case and it wasn’t that easy to set up.

So I figured I should make a new version, written in Python (just to practice it), and so I did.

The new Tespeed is available at Github: https://github.com/Janhouse/tespeed

It is licensed under MIT license.

There are still some bugs and planned features left to sort out, but after testing it on multiple computers it is working fine.

 

Tespeed in automatic mode looking for best testing server and doing download and upload tests.

The new version finds closest servers and then picks the one with lowest latency.

 

Available server list top 25 (by distance)

 

You can also get a list of all available servers (and see the approximate distance to them) by using list-servers command.

 

Manually specified test server.

 

Please send me some feedback so I could make it better.

And go thank speedtest.net for this great service. 🙂

Automatic movie screenshot maker for Auto Uploader

I have finally found some time to make new features for the Auto Uploader.

So, the Thumbnailer is finally here.

Features:

  • Creates video thumbnails (single shots and joined timelapse);
  • Creates thumbnails even if video file is packed inside .rar archive (Linux only);
  • Skips sample files;
  • Extracts basic information about video file (resolution, codec, bitrate, etc.);
  • Automatically uploads created images to public image hosting sites and creates BBCode;
  • Can be fine tuned any way you want.

I am providing this as an external module for the Uploader.

If you want to get it, feel free to contact me. 🙂

 

Generating and uploading screenshots:

Generating screenshots:

I am going to upload a better video once I set up the new version on some tracker.

Following screenshot demonstrates BBCode generated by the Uploader:

Upgrades to announcement system

Some of you know that there are times when release announcements for certain download sources stop appearing for some reason. Mostly it is because the announcer gets banned from the announcement channel or there are some other problems with the feed.
I promised that I would make it possible for you to help with announcing. And so I did!
 🙂

I made some changes so you could help with passing announcements from your box to the tuper-server, that then sends them out to every connected user.
This way you will get the announcements sooner than before (at least for some sources) and they will keep coming even if some boxes get problems.

Those who want to help out, please contact me. I will give your uploader account permission to send release announcements and help you with setting up the necessary scripts.

Also, to make sure that releases get announced only once, I am now storing announcement history, so you will be able to check the history easily.

Vēl nedaudz par Linux

Pirms vairāk kā gada rakstÄ«ju “Mazliet par Linux”. Å oreiz apskatÄ«Å¡u, manuprāt, svarÄ«gākās labās un sliktās izmaiņas.

Joprojām lietoju Archlinux, kas ir superīga distribūcija, kurā jaunumi parādās ātrāk kā citās, bet tā nav domāta pilnīgiem iesācējiem.

Sākšu ar neforšo

Gnome3 un Unity izstrāde

Pirms gada cerējām, ka pieradīsim pie Gnome 3 grafiskās vides, bet nekā. Gnome 3 un Unity joprojām ir lielākā vilšanās, kopš sāku lietot Linux. Daļa pārmigrēja uz XFCE, es lietoju OpenBox (jā, man ļoti pietrūkst noapaļotie logi un smuki vizuālie efekti). 🙁

Vienkāršais un ērtais Gnome 2 grafiskais interfeiss un Compiz efekti bija viens no galvenajiem iemesliem kāpēc daudzi cilvēki sāka lietot Linux uz saviem galda/portatīvajiem datoriem.

Gnome aizbrauca auzās, bet Canonical nopirka galveno Compiz developeri, lai viņš strādātu pie Unity grafiskās vides. Neesmu vēl redzējis Ubuntu lietotāju, kuram patiktu Unity.

Compiz stabilo 0.9 versiju tā arī neesam ieraudzījuši, jo galvenais programmētājs vergo priekš Ubuntu. Smieklīgi, ka Ubuntu lieto nestabilo Compiz  versiju (pat LTS versijā). Jā, protams, tā ir speciāli sapačota, lai viss būtu forši, bet pārējo distribūciju lietotājiem nākas iztikt ar nestabilo Compiz (un tas patiešām ir nestabils). Var jau lietot Ubuntu Compiz versiju ar visiem ielāpiem, bet tad nākas lietot vēl kaudzi citas Ubuntu pakas un tas var sagādāt lielas problēmas.

Skype, Pidgin un citi IM klienti

Šajā lauciņā notiek tas ko varētu saukt par ne-izstrādi (guļoši developeri/pārāk daudz forki vai negatavi risinājumi/putra in general).

Gribi ko vairāk par IRC? Patīkami tas nebūs. 🙂

Es ļoti priecātos, ja kāds komentāros pieminētu  labu alternatīvu Pidgin.

Labās pārmaiņas

Wine progress (un Office prieki)

Cik atceros, Wine vienmēr ir attÄ«stÄ«jies nenormālos tempos. Tiesa, pierodot pie Linux aplikācijām, Wine nākas izmantot pavisam, pavisam reti. Viens no labākajiem pielietojumiem varētu bÅ«t MS Office 2008 darbināšana. Ar Wine darbināts MS Word patērē mazāk RAM kā Libreoffice Writer, un tas ir krietni lietojamāks. LibreOffice pēdējās versijas uzstartējas pāris sekunžu laikā (pirms tam bija ilgi, ilgi jāgaida), bet Word caur Wine (pieņemot, ka fonā darbojas wineserver) – apmēram sekundes laikā.

Patīk, ka Wine vairs lieki nenoslogo CPU, un Ofisā, beidzot, darbojas fontu antialiasing (smoothing).

Testēju gan Word, gan Excel, gan Power Point – darbojās labi. 🙂

Gimp 2.8

Ar nokavēšanos iznākusi jaunā stabilā Gimp versija. Pamanāmākā un, manuprāt, svarÄ«gākā izmaiņa ir “Single window mode”, kas ļauj to lietot vienā logā. Man nav bijuÅ¡i citi iebildumi pret Gimp, tāpēc nevaru komentēt vai ir salabota vēl kāda cita kaitinoÅ¡a problēma.

Citi pozitīvi jaunumi

Jau iepriekÅ¡ rakstÄ«ju, ka uz Linux pieejama alternatÄ«va Photoshop Lightroom – “Corel Aftershot Pro”.

Valve gatavo Steam priekš Linux (būs arī pieejamas Source dziņa spēles).

Kodols attīstās ļoti patīkamos tempos un sen neesmu saskāries ar draiveru problēmām. 🙂

 

Kuru distribūciju lieto un kāpēc? Kā, tavuprāt, attīstās Linux un kā tam vajadzētu attīstīties?

FTP and file feature for auto uploader

Few days ago FTP feature was added to the uploader. It means that any files/folders can be easily added to the uploader’s database. This allows using FTP servers and other sources to auto upload to your tracker (or maybe even upload to some other FTP servers).

So basically you can make it work in any way you want now. FTP to Tracker, Tracker to Tracker, Tracker to FTP, FTP to FTP or even make it work with direct files on the web.

Those of you who already have the access to the new Wiki for the Uploader, can look there for information on setting it up.

I’ll keep making this new FTP thing better so any feedback is welcome in the bugtracker’s forum. 🙂

From the shiny features, automatic screenshot grabber script for videos is left to port to V3.

Dzeltenā e-talona atlikums Android telefonā

Citreiz gribas noskaidrot cik braucieni palikuši dzeltenajā e-talonā, nemeklējot tuvāko e-talona automātu. Protams, negribas arī aplauzties sabiedriskajā, pērkot dārgo biļeti (šogad vēl 70 sant.?).

Uzrakstīju Android aplikāciju, kura ļauj apskatīt dzeltenās kartes saturu un mēģina atšifrēt biļetes veidu un braucienu skaitu.

Dotajā brīdī aplikācija ir īpaši primitīva, negarantēju, ka atšifrējumi ir korekti visos gadījumos (nebija man tik daudz kartes ar kurām testēt). Ar laiku pieslīpēšu un pievienošu kādu jauku fīču (piem., laika biļetes atšifrēšanu?) vai vizuālo noformējumu.

Veidošanā noderēja (daļēji kļūdainā) informācija no Jāņa pierakstiem un citu cilvēku padomiem.

Interesentiem – aplikācija novelkama Android Marketā.

Lai aplikācija darbotos, nepieciešams Android >2.3.3 tālrunis ar NFC.

 

Updeits: Updeitoju .apk, lai saprot derīgo biļešu skaitu jaunajiem e-taloniem. Šķiet, ka Rīgas Satiksme tos bieži maina.

Pieļauju, ka būtu labi savākt daudzus e-talona dumpus vai tikt pie tiem, kuri Jānim bija pieejami pirms pāris gadiem, lai izveidotu maksimāli labu atpazīšanu.

Update 2: Uploadoju aplikāciju uz Android Market, lai būtu vieglāk updeitoties. Darbojas arī vecais links.

 

Android, pērkami bloggeri un emocijas

IzlasÄ«ju Krizdabz emocionālo rakstu, kurā viņš pēc “WINDOWS BUILD” apmeklēšanas sÅ«kstās, kāpēc viņam neiet pie sirds Android OS un iekārtas, uz kurām tas uzstādÄ«ts.

Ikdienā lietoju Nexus S, uz kura darbojas Android 2.3.4. Neskatoties uz to, ka ir manītas dažādas nepilnības, es pilnīgi noteikti neesmu saskāries ar tik daudzām Android ekosistēmas problēmām, tāpēc uzrakstīšu savus komentārus.

Apskatīšu un pakomentēšu viņa raksta apakštēmas.

Continue reading “Android, pērkami bloggeri un emocijas”