Announcement

Collapse
No announcement yet.

programatically select all text in a browser from a button click in a pb app window.

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • programatically select all text in a browser from a button click in a pb app window.

    I knocked together the "after the ctrl/A on the browser" bit a while back (see below)
    i.e. once the text's in the clipboard
    and did try to do the ctrl/a bit in another language but it seemed so slow and unreliable particularly when repeatedly tried in quick succession
    whereas actually clicking on the browser and manually pressing ctrl/A was always fast and fool-proof.
    Why the difference and is it get-around-able?

    i.e. what I'd like to do is...
    1 have a browser open on the page I want
    2 fire-up my little pb app featuring a button and
    3 repeatedly click that button such that the text in the browser is repeatedly written to a file or string for processing
    I'd rather not use some embedded browser control i.e. I'd like this to work with any browser


    Code:
    'http://www.garybeene.com/power/pb-tutor-controls-button.htm
    #COMPILE EXE
    
    #INCLUDE "win32api.inc"
    
    GLOBAL hDlg AS DWORD
    GLOBAL title$
    
    FUNCTION PBMAIN() AS LONG
       title$ = "process clipboard"
       DIALOG NEW PIXELS, 0, title$,300,300,200,200, %WS_SYSMENU, 0 TO hDlg
       CONTROL ADD BUTTON, hDlg, 100,"Push", 50,50,100,20
       CONTROL SET FOCUS hDlg, 100
       DIALOG SHOW MODAL hDlg CALL DlgProc
    END FUNCTION
    
    
    CALLBACK FUNCTION DlgProc() AS LONG
       IF CB.MSG = %WM_COMMAND AND CB.CTL = 100 AND CB.CTLMSG = %BN_CLICKED THEN
          CLIPBOARD GET TEXT TO clp$
          MSGBOX clp$
          create_fl(clp$)
          lres& = ShellExecute(0, "", "perl", "c:\football\do_football.pl ", "", %SW_SHOWNORMAL)
          DIALOG SET TEXT CBHNDL,"done"
          SLEEP 1000
          'lres& = ShellExecute(0, "", "notepad", "c:\football\bets5.txt ", "", %SW_SHOWNORMAL)
          DIALOG SET TEXT CBHNDL,title$
       END IF
    END FUNCTION
    
    
    SUB create_fl(clp$)
      LOCAL i&
      KILL "c:\football\football1.txt"
      OPEN "football1.txt" FOR BINARY AS #1
      SEEK #1, 1 'cos no base specified
      FOR i& = 1 TO LEN(clp$)
        PUT$ #1, MID$(clp$,i&,1)
      NEXT I&
      CLOSE 1
    END SUB

  • #2
    As long it it remains "a" or "any" browser and not a specific brand name and model, it's a crapshoot; setting keyboard focus to the app followed by simulating ctrl+a then ctrl+c to "select all" followed by "copy to the clipboard" has as good a chance as any other of achieving the desired result.
    Michael Mattias
    Tal Systems (retired)
    Port Washington WI USA
    [email protected]
    http://www.talsystems.com

    Comment


    • #3
      >it's a crapshoot
      I was afraid of that and you're not joking i.e. I tried simulating ctrl/a and it "worked" but not well.
      Thank you for the reply anyway....at least it's confirmation that another way would probably be more reliable.

      Comment


      • #4
        ...that another way would probably be more reliable...
        Another way to do what?

        It is not written anywhere you cannot solicit "How to" ideas from other members!

        MCM
        Michael Mattias
        Tal Systems (retired)
        Port Washington WI USA
        [email protected]
        http://www.talsystems.com

        Comment


        • #5

          >It is not written anywhere you cannot solicit "How to" ideas from other members!
          I thought I already had
          >Why the difference and is it get-around-able?

          Comment


          • #6
            ?>It is not written anywhere you cannot solicit "How to" ideas from other members!
            > I thought I already had

            Not really;; you asked for help collecting the text from a browser.

            Unless your application is titled "Let's collect all the text from a browser" then you did not ask for help with the real application.

            That is, "getting the information by copying the text from a browser currently showing the desired text " is a HOW. What you have not shared with us is the WHAT.

            MCM

            Michael Mattias
            Tal Systems (retired)
            Port Washington WI USA
            [email protected]
            http://www.talsystems.com

            Comment


            • #7
              >you have not shared with us is the WHAT.
              Fair enough...
              I want all text here....https://www.betfair.com/sport/inplay
              but they don't like people scraping (otherwise I'd just set wget on it ) hence the ctrl/a in the browser
              Hope that explains

              Comment


              • #8
                Dean, you sure there's not one of the free APIs on betfair that gives you that data? Maybe another provider with developer API like ESPN or MySportsFeeds? Saw somebody mention MySportsFeeds: "If it’s for a non-commercial project, live data is also free. You get unlimited requests against a RESTful API in whatever format you need: XML, JSON, even CSV. Post-game or real-time access is available."

                If you're set on scraping live web page, maybe AutoIT? You could automate IE to go to a page, wait for it to finish loading and do a print or save-as "Webpage Complete" to a file you could parse? Just spit-balling.

                Comment


                • #9
                  Originally posted by Dean Gwilliam View Post
                  >you have not shared with us is the WHAT.
                  Fair enough...
                  I want all text here....https://www.betfair.com/sport/inplay
                  but they don't like people scraping (otherwise I'd just set wget on it ) hence the ctrl/a in the browser
                  Hope that explains
                  Once again, once you know the"what" rather than the "how I thought of doing it", it becomes much easier.

                  This gets that entire page as an HTML document. stripping out the tags and finding the relevant data is left as an exercise for the reader
                  (Hint several code samples for HTML stripping will be found using the forum search function)

                  Code:
                  #INCLUDE "Win32API.inc"
                  #INCLUDE "WinINET.inc"
                  FUNCTION PBMAIN() AS LONG
                     LOCAL URLPath, LocalPath AS WSTRINGZ * %MAX_PATH
                     URLPath = "https://www.betfair.com/sport/inplay"
                     LocalPath = EXE.PATH$ + "test.htm"
                     DeleteURLCacheEntry(URLPath)
                     URLDownloadToFile(BYVAL 0, URLPath, LocalPath, 0, 0)
                      ? "Done"
                  END FUNCTION

                  Comment


                  • #10
                    >one of the free APIs on betfair
                    Yes they have paid for and free but the free one is delayed which means the odds would be out.
                    https://developer.betfair.com/
                    It might do for shots on goal though which I have to get manually at the moment and is a real pain
                    i.e. I'm looking for an increasing rate so realtime wouldn't be essential for that if the delay wasn't great.

                    >If you're set on scraping live web page, maybe AutoIT
                    This was one of the more reliable ways I was alluding to, above, and I did look into it but couldn't really see which commands I needed to make it work.
                    It seems to be a reliable option though according to online opinion.
                    so re
                    >Just spit-balling.
                    No not at all.
                    Another thing I looked at was "mechanize" inside Hutch's recommendation for a browser i.e. slimjet which I use for this.
                    Again, I didn't see how it could be done.

                    Thank you for all of the suggestons,

                    >This gets that entire page as an HTML document
                    That's very cool Stuart and, again, thank you.
                    I had used wget to do this but your all-pb method is very much neater and probably faster
                    One of the things that worries me, though, is is this screen scraping?
                    because I know they take a dim view of it.
                    Having said that if I did it randomly every five or so minutes to detect changes in tempo (which is all I need)
                    might it not look like me just accessing their site via a browser?
                    I don't know enough about the subject and would appreciate some advice on that.

                    Comment


                    • #11
                      I've just used their free api tool to get the games and ctrl/shift/j gives a load of json-looking output.
                      I saw prices but not goals on target and whilst using their front end doesn't give me the automation I'm after
                      ....it's a start
                      Once again....thanks for the suggestions

                      Comment


                      • #12
                        Originally posted by Dean Gwilliam View Post
                        >
                        >If you're set on scraping live web page, maybe AutoIT
                        This was one of the more reliable ways I was alluding to, above, and I did look into it but couldn't really see which commands I needed to make it work.
                        Dean, I actually like getting the page under program control (Stuart's suggestion), but when you said you couldn't use wget, I figured it was a non-started.

                        Below is a little AutoIT script I use to retrieve a data file from the Federal Government. I sanitized it down to what I thought you'd probably need. I too couldn't use wget because you have to interact with several predecessor pages before this point (like Paul's example in another post). Replace the URL and local directory/filename and I think it will get you started.

                        Filename: capture_page.au3
                        Code:
                        #include <IE.au3>
                        Local $oIE = _IECreate("https://www.website.com/page.txt")
                        _IELoadWait($oIE)
                        Send("{ALT}FA")
                        WinWait("Save Webpage")
                        If Not WinActive("Save Webpage","") Then WinActivate("Save As","")
                        ControlSetText ("Save Webpage", "", "Edit1", "F:\directory\filename.txt")
                        Sleep(100)
                        ControlFocus ("Save Webpage", "", "Button1")
                        Send("{enter}")
                        Sleep(2000)
                        Send("{ALT}FC")
                        _IEQuit($oIE)
                        I believe instead of saving the HTML and parsing it, you could replace the Send("(ALT)FA") with your copy/paste idea (Ctrl-A, Ctrl-C), open notepad and Paste (Ctrl-V). I'm probably not far enough through my first pot of coffee to reliably code yet. But if you need pointers just ask. Good luck using whichever suggestion works for you!

                        Comment


                        • #13
                          >I actually like getting the page under program control (Stuart's suggestion), but when you said you couldn't use wget, I figured it was a non-starter.
                          I might give it a go with e.g. 4mins +-RND(a minute or so) and see what happens

                          >I'm probably not far enough through my first pot of coffee to reliably code yet.
                          I know what you mean.

                          Thanks very much Raymond and for the further offer of advice. It's really motivating to have a solid planB and I've heard good things about AutoIT.

                          Comment


                          • #14
                            Dean,

                            There may be another trick, if you start HTMLHELP with a URL, it displays pages OK and as this is geared to access data, it may be easier to semi automate than a fully set up browser.
                            hutch at movsd dot com
                            The MASM Forum

                            www.masm32.com

                            Comment


                            • #15
                              Once again, once you know the"what" rather than the "how I thought of doing it", it becomes much easier.
                              Be Still My Heart! After only twenty years or so of preaching, someone finally "gets it."
                              Michael Mattias
                              Tal Systems (retired)
                              Port Washington WI USA
                              [email protected]
                              http://www.talsystems.com

                              Comment


                              • #16
                                >if you start HTMLHELP with a URL...
                                Hutch.
                                That one wasn't on my radar at all and sounds quite accessible so...thank you very much for it.

                                Michael
                                Thank you

                                Comment


                                • #17
                                  Just pointed the demo program I posted at https://forum.powerbasic.com/forum/u...-api-functions at the URL you provided and it seemed to download the page OK. You need to increase the buffer size from 400,000 to 900,000 bytes though, and it doesn't correctly report the file size in advance of downloading.
                                  - LJ

                                  Comment


                                  • #18
                                    Laurence
                                    I can see that a lot of work has gone into that.
                                    I changed 400000 to 900000 in the two places that search found and then a million when I saw the url is about 970,000 bytes now after running your program once.
                                    I was expecting to see the live football matches which is what you get if you type that url straight into a browser.
                                    For some reason after saving the result of your program as an html file and clicking on it...The browser did show the betfair site but not a live game in site
                                    I'm afraid I have no internet acumen whatsoever to interpret what's going on but just thought I'd let you know.
                                    Irrespective, thank you very much for what looks like an extremely useful, and polished, program.

                                    Comment


                                    • #19
                                      FWIW, in the handy bit of code from Stuart in post #9 of this thread, I was getting a Parameter Mismatch error on line #7.
                                      I eventually solved it by changing the variable declaration in line #4 from WSTRINGZ to ASCIIZ.
                                      Code:
                                      #INCLUDE "Win32API.inc"
                                      #INCLUDE "WinINET.inc"
                                      FUNCTION PBMAIN() AS LONG
                                         LOCAL URLPath, LocalPath AS ASCIIZ * %MAX_PATH
                                         URLPath = "https://www.betfair.com/sport/inplay"
                                         LocalPath = EXE.PATH$ + "test.htm"
                                         DeleteURLCacheEntry(URLPath)
                                         URLDownloadToFile(BYVAL 0, URLPath, LocalPath, 0, 0)
                                          ? "Done"
                                      END FUNCTION
                                      Thanks Stuart!
                                      Rod
                                      I want not 'not', not Knot, not Knott, not Nott, not knot, not naught, not nought, but aught.

                                      Comment


                                      • #20
                                        . . . solved it by changing the variable declaration in line #4 from WSTRINGZ to ASCIIZ.
                                        You've got PBWin 9, or earlier???
                                        Dale

                                        Comment

                                        Working...
                                        X