Announcement

Collapse

Forum Guidelines

This forum is for finished source code that is working properly. If you have questions about this or any other source code, please post it in one of the Discussion Forums, not here.
See more
See less

Convert IP address to geographic location (city, region, country)

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • PBCC Convert IP address to geographic location (city, region, country)

    There's a small handful of IP-to-geolocation database companies, generally however you need to pay a fair amount per year to receive 12 monthly database updates.

    However there is a free database, also updated monthly, called GeoLiteCity, available at http://www.maxmind.com/app/geolitecity
    GeoLite City is the free version of their GeoIP City product (which you have to pay for).

    Here's a basic comparison ...
    Code:
              GeoLite City                      GeoIP City
    Cost      FREE                              $370 initial, $90/mo
    Coverage  Worldwide                         Worldwide
    Accuracy  Over 99.5% on a country level     Over 99.8% on a country level
              and 79% on a city level for the   and 83% on a city level for the
              US within a 25 mile radius.       US within a 25 mile radius.
    Redist.   Free, subject to GPL/LPGL for     Contact us.
              APIs and database license.
              Commercial redistribution
              licenses are available.
    Updates   Monthly, beginning of month.      Monthly, but weekly for binary format.
    So i wrote a little program to utilise the free database. (You can probably pay the $90/mo if you desire and use the paid databases instead as I'm pretty sure they're the same format as the free one, just with a bit more data).

    It's in two parts ... a CONVERTER which takes the original CSV-format databases as downloaded off the maxmind.com website and converts them into a more compact, optimised, memory-efficient version. The conversion generally takes less than 10 seconds, and you only need to do it whenever you download the latest databases (so, once a month max).

    The second part is the READER, which loads the optimised databases and then allows you to input an IP address and it will then return the geographic location.

    Here's a sample output result ...
    IP Address: 127.0.0.1 <- localhost
    Reserved/local
    IP Address: 67.17.204.214
    <- powerbasic.com
    Country: US Region: FL City: Venice


    Because the optimised database is essentially just a list of DWORDs the search itself is very, very quick - practically instant.

    To get started ...
    1. Download the latest CSV FORMAT database from http://www.maxmind.com/app/geolitecity (there's a direct link to a .zip file on that page). I think the zip file is about 15mbs all up, but expands to approximately 150mbs.

    2. Extract the two database files from the zip into the same directory where you'll be running converter.exe and reader.exe from.

    3. Compile and run the converter.bas program, this will create the optimized database (ipcities.dat and ipranges.dat), which are about 50mbs all up. This is what the reader loads into memory. (You're free to delete the existing .csv databases at this stage if you want that 150mbs disk space back)

    4. Now compile and run the reader.bas program, give it any IP address and it'll return the geographic IP location.

    Just a couple notes ... in some cases it may not be able to return a city location but can still return the country location, so never assume it will always be able to determine the city or region. Also, the reader expects a numeric IP address, ie. one that has already been resolved - to keep things as bare-bones as possible i didn't add any DNS lookup code, though that'd only be a few extra lines.

    The country returned by the database is simply a country code (ie. "AU") ... to convert those codes to full country names ie Australia see this function

    ---

    The source code is ready-to-compile in PBCC.
    To compile in PBWIN should simply be a matter of replacing STDOUT with MSGBOX, removing WAITKEY$ references, and changing the STDIN LINE reference to INPUTBOX$ or similar at the stage where it asks you the user for an IP address.

    CONVERTER.BAS
    Code:
    #COMPILE EXE
     
    TYPE IPRange
     dwStart AS DWORD
     dwEnd   AS DWORD
     dwArea  AS DWORD
    END TYPE
     
    FUNCTION PBMAIN() AS LONG
    LOCAL sBuf AS STRING, cnt AS DWORD, i AS DWORD, i2 AS DWORD, sLine AS STRING, x AS DWORD, x2 AS DWORD, IPR AS IPRange, sTmp AS STRING
     
    DOBLOCKS:
    IF DIR$("geolitecity-blocks.csv",39) = "" THEN
        STDOUT "Warning, geolitecity-blocks.csv not found": WAITKEY$: EXIT FUNCTION
    END IF
    STDOUT "Reading 1/2 ... ";
    OPEN "geolitecity-blocks.csv" FOR BINARY ACCESS READ AS #1
     GET$ #1, LOF(1), sBuf
    CLOSE #1
    KILL "ipranges.dat"
    OPEN "ipranges.dat" FOR BINARY ACCESS WRITE AS #1
    STDOUT $CRLF & "Processing 1/2 ... ";
    i = INSTR(1, sBuf, CHR$(&hA)): i = INSTR(i+1, sBuf, CHR$(&hA))
    DO
     i = i + 1
     IF i => LEN(sBuf) THEN EXIT DO
     i2 = INSTR(i, sBuf, CHR$(&hA))
     IF i2 = 0 THEN EXIT DO
     sLine = MID$(sBuf, i + 1, i2 - i - 1)
     x = 1
     x2 = INSTR(x+1,sLine, CHR$(34))
     IPR.dwStart = VAL(MID$(sLine, x, x2 - x))
     x = x2 + 3
     x2 = INSTR(x+1, sLine, CHR$(34))
     IPR.dwEnd = VAL(MID$(sLine, x, x2 - x))
     x = x2 + 3
     x2 = INSTR(x+1, sLine, CHR$(34))
     IPR.dwArea = VAL(MID$(sLine, x, x2 - x))
     i = i2
     INCR cnt
     sTmp = PEEK$(VARPTR(IPR), 12)
     PUT$ #1, sTmp
    LOOP
    i = LOF(1)
    CLOSE #1
    STDOUT $CRLF & "Done! " & TRIM$(STR$(cnt)) & " IP Ranges saved. Filesize " & FORMAT$(i / 1024 / 1024, "0.0") & "mbs"
    
    DOAREAS:
    LOCAL dwCode AS DWORD, sCountry AS STRING, sRegion AS STRING, sCity AS STRING, dwHigh AS DWORD
    IF DIR$("geolitecity-blocks.csv",39) = "" THEN
        STDOUT "Warning, geolitecity-blocks.csv not found": WAITKEY$: EXIT FUNCTION
    END IF
    cnt=0
    STDOUT "Reading 2/2 ... ";
    OPEN "geolitecity-location.csv" FOR BINARY ACCESS READ AS #1
     GET$ #1, LOF(1), sBuf
    CLOSE #1
    KILL "ipcities.dat"
    OPEN "ipcities.dat" FOR BINARY ACCESS WRITE AS #1
    STDOUT $CRLF & "Processing 2/2 ... ";
    i = INSTR(1, sBuf, CHR$(&hA)): i = INSTR(i+1, sBuf, CHR$(&hA))
    DO
     i = i + 1
     IF i => LEN(sBuf) THEN EXIT DO
     i2 = INSTR(i, sBuf, CHR$(&hA))
     IF i2 = 0 THEN EXIT DO
     sLine = MID$(sBuf, i, i2 - i - 1)
     x = INSTR(1, sLine, ",")
     dwCode = VAL(LEFT$(sLine, x - 1))
     IF dwCode > dwHigh THEN dwHigh = dwCode
     x = x+1
     x2 = INSTR(x, sLine, ",")
     sCountry = MID$(sLine, x, x2 - x)
     IF LEFT$(sCountry,1) = CHR$(34) THEN sCountry = RIGHT$(sCountry, LEN(sCountry) - 1)
     IF RIGHT$(sCountry,1) = CHR$(34) THEN sCountry = LEFT$(sCountry, LEN(sCountry) - 1)
     x = x2+1
     x2 = INSTR(x, sLine, ",")
     sRegion = MID$(sLine, x, x2 - x)
     IF LEFT$(sRegion,1) = CHR$(34) THEN sRegion = RIGHT$(sRegion, LEN(sRegion) - 1)
     IF RIGHT$(sRegion,1) = CHR$(34) THEN sRegion = LEFT$(sRegion, LEN(sRegion) - 1)
     x = x2+1
     x2 = INSTR(x, sLine, ",")
     sCity = MID$(sLine, x, x2 - x)
     IF LEFT$(sCity,1) = CHR$(34) THEN sCity = RIGHT$(sCity, LEN(sCity) - 1)
     IF RIGHT$(sCity,1) = CHR$(34) THEN sCity = LEFT$(sCity, LEN(sCity) - 1)
     i = i2
     sTmp = PEEK$(VARPTR(dwCode),4) & sCountry & CHR$(0) & sRegion & CHR$(0) & sCity & CHR$(0)
     sTmp = CHR$(LEN(sTmp)) & sTmp
     PUT$ #1, sTmp
     INCR cnt
    LOOP
    i = LOF(1)
    CLOSE #1
    STDOUT $CRLF & "Done! " & TRIM$(STR$(cnt)) & " locations saved. Filesize " & FORMAT$(i / 1024 / 1024, "0.0") & "mbs"
    
    STDOUT "Database conversion complete, ipcities.dat and ipranges.dat are ready for use."
    WAITKEY$
    END FUNCTION

    READER.BAS
    Code:
    #COMPILE EXE
    DECLARE FUNCTION inet_addr LIB "ws2_32.dll" ALIAS "inet_addr" (cp AS ASCIIZ) AS DWORD
    
    FUNCTION ShowLocation(BYVAL dwLoc AS DWORD, sBuf AS STRING) AS STRING
    LOCAL bPtr AS BYTE PTR, dwPtr AS DWORD PTR, szPtr AS ASCIIZ PTR, sOut AS STRING, sTmp AS STRING
    bPtr = STRPTR(sBuf)
    DO
        dwPtr = bPtr + 1
        IF @dwPtr = dwLoc THEN
            szPtr = dwPtr + 4
            IF LEN(@szPtr) > 0 THEN sOut = "Country: " & @szPtr
            szPtr = szPtr + LEN(@szPtr) + 1
            IF LEN(@szPtr) > 0 THEN sOut = sOut & " Region: " & @szPtr
            szPtr = szPtr + LEN(@szPtr) + 1
            IF LEN(@szPtr) > 0 THEN sOut = sOut & " City: " & @szPtr
            FUNCTION = sOut
            EXIT FUNCTION
        END IF
        bPtr = bPtr + @bPtr + 1
    LOOP
    END FUNCTION
    
    
    FUNCTION PBMAIN() AS LONG
    LOCAL sPath AS STRING, sBuf AS STRING, sLocBuf AS STRING, i AS DWORD, i2 AS DWORD, cnt1 AS DWORD, hFile AS DWORD
    LOCAL sIP AS STRING, szIP AS ASCIIZ * 255, dwIP AS DWORD, dwPtr AS DWORD PTR, dwPtr2 AS DWORD PTR, dwLoc AS DWORD, dwEnd AS DWORD
    
    IF DIR$("ipcities.dat",39) = "" OR DIR$("ipranges.dat",39) = "" THEN
        STDOUT "Error: ipcities.dat/ipranges.dat not found": WAITKEY$
        EXIT FUNCTION
    END IF
    
    STDOUT "Loading databases ... ";
    hFile = FREEFILE
    OPEN "ipranges.dat" FOR BINARY ACCESS READ AS #hFile
     GET$ #hFile, LOF(hFile), sBuf
    CLOSE #hFile
    hFile = FREEFILE
    OPEN "ipcities.dat" FOR BINARY ACCESS READ AS #hFile
     GET$ #hFile, LOF(hFile), sLocBuf
    CLOSE #hFile
    STDOUT "OK"
    
    DO
        STDOUT "IP Address: ";
        STDIN LINE szIP
        dwIP = inet_addr( szIP )
        ! mov eax, dwIP
        ! bswap eax
        ! mov dwIP, eax
        IF dwIP => 167772160 AND dwIP <= 184549375 THEN         '10.0.0.0 = 167772160      10.255.255.255 = 184549375
           ReservedIP:
            STDOUT "Reserved/local"
            ITERATE
        ELSEIF dwIP => 2130706432 AND dwIP <= 2147483647 THEN  '127.0.0.0   = 2130706432   127.255.255.255 = 2147483647
            GOTO ReservedIP
        ELSEIF dwIP => 2886729728 AND dwIP <= 2887778303 THEN  '172.16.0.0  = 2886729728   172.31.255.255  = 2887778303
            GOTO ReservedIP
        ELSEIF dwIP => 3232235520 AND dwIP <= 3232301055 THEN  '192.168.0.0 = 3232235520   192.168.255.255 = 3232301055
            GOTO ReservedIP
        END IF
        dwLoc = 0
        dwPtr = STRPTR(sBuf)
        dwEnd = dwPtr + LEN(sBuf) - 10
        DO
            dwPtr2 = dwPtr + 4
            IF dwIP => @dwPtr AND dwIP <= @dwPtr2 THEN
                dwPtr2 = dwPtr2 + 4
                dwLoc = @dwPtr2
                EXIT DO
            END IF
            dwPtr = dwPtr + 12
            IF dwPtr => dwEnd THEN EXIT DO
        LOOP
        IF dwLoc = 0 THEN
            STDOUT "Unknown"
        ELSE
            STDOUT ShowLocation(BYVAL dwLoc, sLocBuf)
        END IF
    LOOP
    
    END FUNCTION
    License
    Under the license agreement, all advertising materials and documentation mentioning features or use of this database must display the following acknowledgment: "This product includes GeoLite data created by MaxMind, available from http://www.maxmind.com/."
    For more info see http://geolite.maxmind.com/download/...se/LICENSE.txt
    Last edited by Wayne Diamond; 2 Aug 2009, 03:53 AM.
    -

  • #2
    There's also a free online database at http://www.ipinfodb.com which uses the MaxMind free database. You can retrieve an entry with this code - based on the Help file TCP Open example.

    Parsing the XML content is left as an exercise for the reader

    Code:
    #COMPILE EXE
    #DIM ALL
    FUNCTION PBMAIN() AS LONG
      LOCAL Buffer$, Site$
      LOCAL Entire_page$, Htmlfile$, Link$
      LOCAL Pos&, Length&,File$
      LOCAL strIP AS STRING
      strIP = "67.17.204.214"
      Site$ = "www.ipinfodb.com"
      File$ = "http://www.ipinfodb.com/ip_query.php?ip=" & strIP & "&output=xml"
      ' Connecting...
      TCP OPEN "http" AT Site$ AS #1 TIMEOUT 60000
      ' Could we connect to site?
      IF ERR THEN
          BEEP
          EXIT FUNCTION
      END IF
      ' Send the GET request...
      TCP PRINT #1, "GET " & File$ & " HTTP/1.0"
      TCP PRINT #1, "Referer: http://www.powerbasic.com/"
      TCP PRINT #1, "User-Agent: Get IP GeoLocation Example (www.powerbasic.com)
      TCP PRINT #1, ""
      ' Retrieve the page...
      DO
        TCP RECV #1, 4096, Buffer$
        Entire_page = Entire_page + Buffer$
      LOOP WHILE ISTRUE LEN(Buffer$) AND ISFALSE ERR
      ' Close the TCP/IP port...
      TCP CLOSE #1
      MSGBOX Entire_page
    END FUNCTION

    Comment


    • #3
      Online lookup? But that's cheating ...

      Another online option is to simply use the Maxmind database directly (limited to 25 lookups per day though)

      Code:
      #COMPILE EXE
       
      DECLARE FUNCTION URLDownloadToFile LIB "urlmon.dll" ALIAS "URLDownloadToFileA" _
              (BYVAL pCaller AS DWORD, szURL AS ASCIIZ, szFileName AS ASCIIZ, _
               BYVAL dwReserved AS DWORD, BYVAL fnCB AS DWORD) AS LONG
        
      FUNCTION PBMAIN() AS LONG
      LOCAL sIP AS STRING
      sIP = "67.17.204.214"
      IF URLDownloadToFile(BYVAL 0, "http://www.maxmind.com/app/lookup_city?ips=" & sIP, "c:\ip.htm", BYVAL 0, BYVAL 0) = 0 THEN
          STDOUT "OK, see c:\ip.htm"
      END IF
      END FUNCTION
      -

      Comment


      • #4
        DECLARE FUNCTION URLDownloadToFile LIB "urlmon.dll" ?

        D*mn, I should have known that there was a simple API call for that

        Comment


        • #5
          Originally posted by Wayne Diamond View Post
          Online lookup? But that's cheating ...

          Another online option is to simply use the Maxmind database directly (limited to 25 lookups per day though)

          Code:
          #COMPILE EXE
           
          DECLARE FUNCTION URLDownloadToFile LIB "urlmon.dll" ALIAS "URLDownloadToFileA" _
                  (BYVAL pCaller AS DWORD, szURL AS ASCIIZ, szFileName AS ASCIIZ, _
                   BYVAL dwReserved AS DWORD, BYVAL fnCB AS DWORD) AS LONG
           
          FUNCTION PBMAIN() AS LONG
          LOCAL sIP AS STRING
          sIP = "67.17.204.214"
          IF URLDownloadToFile(BYVAL 0, "http://www.maxmind.com/app/lookup_city?ips=" & sIP, "c:\ip.htm", BYVAL 0, BYVAL 0) = 0 THEN
              STDOUT "OK, see c:\ip.htm"
          END IF
          END FUNCTION
          Ahem, Avira states that this program, when compiled, is a virus...

          Description:
          HEUR/Malware



          HEUR/Malware is a heuristic detection routine designed to detect common malware characteristics. Avira AntiVir recognizes unknown malware proactively using its AHeAD technology. To achieve this, Avira performs innovative structural analyzing.

          On the basis of the composition of a file, the sequence of significant code sequences or based on particular behavior patterns, the heuristics can determine with a high probability whether it is dealing with a harmful or virulent file.

          HEUR/Malware in particular is reported when a program seems to contain suspicious functionality.

          In the unlikely occurrence of a false positives we would kindly ask for your help and send the file to our virus lab using the quarantine functionality of AntiVir.

          A heuristic detection might be a false identification if one or more of the following are true:
          - The program has been used for a very long time and is known to the user
          - The program was installed by the user himself
          - The program is from a trustworthy source

          Please note that even old programs can get infected or replaced by malware without your knowledge. Besides that trustworthy sources might have become compromised themselves.

          In order to enhance detection and reduce the rate of false positives we recommend you to send the file to our virus lab for further analysis.
          Andrea Mariani
          AS/400 expert
          Basic programmer @ Home

          Comment


          • #6
            Time to change of antivirus.
            Forum: http://www.jose.it-berater.org/smfforum/index.php

            Comment

            Working...
            X