Video Capture Redux

[Gary Beene]

Well, my interests have once again focused on capturing video from an attached camera.

I've previously used VFW, which is very clunky and hard to use when more than 1 camera is attached. It just can't see some cameras.

I've had someone build me a C# app using AForge, written so that I can control it from my own PowerBASIC app. It works, but just doesn't have as clean an interface and isn't as responsive as I'd like.

I've done a similar thing with ffmpeg - create a window that I control from a PowerBASIC app. The whole idea of another window, controlled by my PowerBASIC app, works but never seems to give a well-integrated look.

My web search often turns up a number of .NET approaches, none of which help me out with viewing the camera in a PowerBASIC app.

Terminology from my searching ...
VFW
MediaCapture
CameraCaptureUI
LaunchURIAsync
WIA
DirectShow
OpenCV
AForge
HTML5 getUserMedia
Win10 Camera app
ffmpeg
LibVLC
Mlultimedia Streaming
VIdeoLAN (VLC)

I guess of all that I've read, DirectShow seems to be the most compatible with PowerBASIC but also widely disparaged as poorly documented and as having a steep learning curve.

I'm pretty sure that I've spoken via PM (a few years back) about DirectShow to someone here on the forum, but I don't see any examples to help me get started.

If anyone has guidance to offer please speak up!

[Russ Srole]

Gary,

If this is the same project we spoke about a year or so ago, then perhaps I can help. I haven't worked with all of the above technologies, but I have some experience with directshow in c#. So with that limited knowledge base, I agree that directshow is probably your best bet. It is awful in it's own way, but it will allow you to capture the video, run it through a filter called a "samplegrabber", modify the video as you did in the c# app we played with, and then display it. I'm on vacation now, so I'm not inclined to take time right now, but maybe next week or the weekend after this one, I can put together a little tutorial. If you can wait then I'd be glad to help out.

Russ

[Gary Beene]

Hi Russ!
Yep, same project. Just trying to integrate the video into my PowerBASIC app.

Thank you for the offer and I'll look forward to hearing from you after vacation!

I'll spend time in the interim trying to get a handle on the technology - at least enough to get the most out of your tutorial!

[Gary Beene]

In Jose's sub-forum is an example for displaying a video file using DirectShow. Here's an extraction of the minimal code that gets the job done. This kind of minimal code helps me understand the concepts better.

In this example, the dialog is used to display the video. It's hard wired to run the video file "bubbles.mov". When the dialog is resized, the video display is resized to fit the dialog client area, maintaining the aspect ratio of the original video.

I ran a few tests and was able to use a Label and a Graphic Control as the display window, but when resizing, both of those seem to cause flicker so I stuck with displaying the video on the Dialog itself.

'Compilable Example:  Play Video - Minimal Code
#Compiler PBWin 10
#Compile Exe
#Dim All
%Unicode = 1
#Include "Win32API.inc"
#Include Once "dshow.inc"
#Include Once "ole2utils.inc"   ' For IUnknown_Release

%IDC_Graphic = 500
%WM_GraphNotify = %WM_User+13

Global hDlg As Dword

' Interface pointers
Global pIGraphBuilder As IGraphBuilder
Global pIMediaControl As IMediaControl
Global pIMediaEventEx As IMediaEventEx
Global pIVideoWindow  As IVideoWindow

Function PBMain() As Long
   Dialog New Pixels, 0, "Direct Show Tests",,,600,400, %WS_OverlappedWindow To hDlg
   Dialog Show Modal hDlg Call DlgProc
End Function

CallBack Function DlgProc() As Long
   Local w,h As Long, rc As Rect
   Select Case Cb.Msg
      Case %WM_InitDialog
         PlayMovieInWindow(hDlg, "bubbles.mov")
      Case %WM_Size
         GetClientRect hDlg, rc
         If IsObject(pIVideoWindow) Then
            pIVideoWindow.SetWindowPosition(rc.Left, rc.Top, rc.Right, rc.Bottom)
            RedrawWindow hDlg, rc, 0, %RDW_INVALIDATE Or %RDW_UPDATENOW
         End If
   End Select
End Function

Sub PlayMovieInWindow (ByVal hwnd As Dword, ByRef wszFileName As WStringZ)
   Local hr As Long
   pIGraphBuilder = NewCom ClsId $CLSID_FilterGraph   ' Create an instance of the IGraphBuilder object
   pIMediaControl = pIGraphBuilder                             ' Get reference pointers
   pIMediaEventEx = pIGraphBuilder
   pIVideoWindow  = pIGraphBuilder
   hr = pIGraphBuilder.RenderFile(wszFileName)        ' Render the file
   pIVideoWindow.Visible = %OATRUE                        ' Set the window properties
   pIVideoWindow.Owner = hwnd
   pIVideoWindow.WindowStyle = %WS_Child Or %WS_ClipSiblings Or %WS_ClipChildren
   pIMediaControl.Run                                                    ' Run the graph
End Sub

In summary, the application exposes 4 Filter Graph Manager interfaces which provide the methods needed in this app to read and display the video file.

My goal is to understand DirectShow well enough to be able to select and display a camera. What I've read so far says the ICaptureGraphBuilder2 interface will be required. I'll post more as I understand it better.

[Gary Beene]

Well, after not making progress on using DirectShow for the last two years, I finally put in some extra time and got some (minimal) results.

This short code uses DirectShow to iterate through the video devices on my PC ....

#Compile Exe
#Dim All
%Unicode = 1
#Include "Win32API.inc"
#Include Once "dshow.inc"
Function PBMain() As Long
   Local pSysDevEnum  As ICreateDevEnum
   Local pEnumCat     As IEnumMoniker
   Local pMoniker     As IMoniker
   Local hr           As Dword
   Local pwszDisplayName As WStringZ Ptr

   pSysDevEnum = NewCom ClsId $CLSID_SystemDeviceEnum
   hr = pSysDevEnum.CreateClassEnumerator($CLSID_VideoInputDeviceCategory, pEnumCat, 0)
   If hr = %S_Ok Then ? "Success"
   Do
      hr = pEnumCat.next(1, pMoniker, pceltFetched)
      hr = pMoniker.GetDisplayName(pbc, Nothing, pwszDisplayName)
      ? @pwszDisplayName
      If hr = %S_False Then Exit Do
   Loop
End Function

It's hardly a monument of success - it won't exit the loop and it doesn't give the user-friendly name of the cameras. But even so, I am tickled to have made any progress - however partial it might be!

I've read that the IMONIKER::BINDTOSTORAGE method needs to be used to get the friendly name of the devices. I'll work on that next. That's all a prelude to actually displaying a live video using DirectShow.

[José Roca]

The check for %S_FALSE is misplaced. Change it to:

Do
   hr = pEnumCat.next(1, pMoniker, pceltFetched)
   If hr = %S_False Then Exit Do
   hr = pMoniker.GetDisplayName(pbc, Nothing, pwszDisplayName)
   ? @pwszDisplayName
Loop

To get the friendly name, you must do something like

DIM pPropBag AS IPropertyBag
hr = pMoniker.BindToStorage(NOTHING, NOTHING, $IID_IPropertyBag, pPropBag)
IF SUCCEEDED(hr) THEN
   DIM varName AS VARIANT
   hr = pPropBag.Read("FriendlyName", varName, NOTHING)
   PRINT VARIANT$$(varName)
END IF

[Gary Beene]

Howdy, Jose!
Works like a charm. I'm glad to know that I wasn't on a totally wrong path. Thanks for stepping in and helping!

The complete example for enumerating video devices using DirectShow ...

'Compilable Example:
#Compiler PBWin 10
#Compile Exe
#Dim All
%Unicode = 1
#Include "Win32API.inc"
#Include Once "dshow.inc"
Function PBMain() As Long
   Local pSysDevEnum  As ICreateDevEnum
   Local pEnumCat     As IEnumMoniker
   Local pMoniker     As IMoniker
   Local hr           As Dword
   Local pwszDisplayName As WStringZ Ptr
   Local pceltFetched As Dword
   Local pbc          As IBindCTX
   Local pPropBag     As IPropertyBag
   Local varName      As Variant
   Local DeviceList$

   'create the enumeration class
   pSysDevEnum = NewCom ClsId $CLSID_SystemDeviceEnum
   hr = pSysDevEnum.CreateClassEnumerator($CLSID_VideoInputDeviceCategory, pEnumCat, 0)
   'now iterate the list and get the names of the attached video devicese
   Do
      hr = pEnumCat.next(1, pMoniker, pceltFetched)
      If hr = %S_False Then Exit Do
      hr = pMoniker.GetDisplayName(pbc, Nothing, pwszDisplayName)  'pwszDisplayName is name complex camera name
      'now get the "Friendly Name"
      hr = pMoniker.BindToStorage(Nothing, Nothing, $IID_IPropertyBag, pPropBag)
      If SUCCEEDED(hr) Then
         hr = pPropBag.Read("FriendlyName", varName, Nothing)
         DeviceList$ += Variant$$(varName) + $CrLf
      End If
   Loop
   ? DeviceList$
End Function

[Paul Purvis]

Gary, you might pick up some help in the "Autoit" ream of things off the internet.
Of course no promises you will but that program has really changed enough and seems close in coding to some PB needed coding that might not be found in the history of this forum.

[Gary Beene]

Hey Paul!
Do you mean that the AutoIt forum might have information about DirectShow? I went to the site and did not quickly find anything that caught my attention. But, I did go into the forum and saw several threads regarding DirectShow.. I'll have to delve into the links more to see if there's content that might help out.

[Paul Purvis]

Just general searches
i searched autoit and directshow then came across what seemed helpful. Also replace directshow word with webcam

[Paul Purvis]

Gary, I have no idea whether this can help or not.
It was referenced thru an autoit search.
http://sol.gfxile.net/escapi/

[Gary Beene]

Hi Paul!

Yes, thanks, I saw that too. It's not unlike VideoCapX, where a DLL has been created to provide Direct Show video capture capabilities.

I've made a note of both of those DLLs, but for now would prefer to figure out how to implement video device selection/preview using DirectShow API from within my app.

I've made some, but frustratingly slow progress over the last couple of days, so I'll keep at it for a while before considering the DLL options.

[Paul Purvis]

Gary, I feel the same way about my programs, I fully understand, specially on a DLL. that is not a standard windows install.

[Gary Beene]

I don't have success to report - success being defined as selecting a video device and Previewing it in a dialog using DirectShow.

But, I've read through the MS Video Capture section a dozen times have managed to follow their suggestions at least create a beginning of the solution.

Interestingly, the solution is turning out to be just a handful of lines of code more than the code from #7 above. You'd think a handful of lines of code would not be hard to make work, but it's been a humbling experience.

Here's the work in progress. Please take a look and let me know if you have any suggested improvements.

#Compile Exe
#Dim All
#Debug Error On
#Debug Display On
%Unicode = 1
#Include "Win32API.inc"
#Include Once "dshow.inc"
#Include Once "ole2utils.inc"   ' For IUnknown_Release

%WM_GraphNotify = %WM_User+13

Global hDlg As Dword
Global pWindow As IVideoWindow            'Display Window

Function PBMain() As Long
   Dialog New Pixels, 0, "Direct Show - Video Preview",,,600,400, %WS_OverlappedWindow To hDlg
   Dialog Show Modal hDlg Call DlgProc
End Function

CallBack Function DlgProc() As Long
   Local w,h As Long, rc As Rect
   Select Case Cb.Msg
      Case %WM_InitDialog
         DisplayVideo hDlg, "Logitech HD Pro Webcam C920"
      Case %WM_Size
         GetClientRect hDlg, rc
         If IsObject(pWindow) Then
            pWindow.SetWindowPosition(rc.Left, rc.Top, rc.Right, rc.Bottom)
            RedrawWindow hDlg, rc, 0, %RDW_INVALIDATE Or %RDW_UPDATENOW
         End If
   End Select
End Function

Sub DisplayVideo (ByVal hwnd As Dword, DeviceName$)
   Local pGraph          As IGraphBuilder           'Filter Graph Manager
   Local pBuild          As ICaptureGraphBuilder2   'Capture Graph Builder
   Local pSysDevEnum     As ICreateDevEnum          'enumeration object
   Local pEnumCat        As IEnumMoniker
   Local pMoniker        As IMoniker                'contains information about other objects
   Local pCap            As IBaseFilter             'Video capture filter
   Local hr              As Dword
   Local pwszDisplayName As WStringZ Ptr
   Local pceltFetched    As Dword
   Local pbc          As IBindCTX
   Local pPropBag        As IPropertyBag
   Local varName         As Variant


   'necessary objects
   pGraph      = NewCom ClsId $CLSID_FilterGraph          'filter graph
   pBuild      = NewCom ClsId $CLSID_CaptureGraphBuilder2 'capture graph builder
   pSysDevEnum = NewCom ClsId $CLSID_SystemDeviceEnum     'enumeration object

   'get the Moniker whose friendly name matches DeviceName$
   hr = pSysDevEnum.CreateClassEnumerator($CLSID_VideoInputDeviceCategory, pEnumCat, 0)  'list of monikers for video sources
   Do   'iterate the list until the moniker friendly name matches the chosen device
      hr = pEnumCat.next(1, pMoniker, pceltFetched)
      If hr = %S_False Then Exit Do
      hr = pMoniker.GetDisplayName(pbc, Nothing, pwszDisplayName)                 'get complex camera name
      hr = pMoniker.BindToStorage(Nothing, Nothing, $IID_IPropertyBag, pPropBag)  'get info about Moniker
      If SUCCEEDED(hr) Then
         hr = pPropBag.Read("FriendlyName", varName, Nothing)   'get friendly name
         If Variant$$(varName) = DeviceName$ Then Exit Do       'this Moniker matches the one of interst
      End If
   Loop

   hr = pMoniker.BindToObject(Nothing, Nothing, $IID_IBaseFilter, pCap)                  'create device filter for the chosen device
   hr = pGraph.AddFilter(pCap,"Logitech HD Pro Webcam C920")                             'add chosen device filter to the filter graph
   hr = pBuild.SetFilterGraph(pGraph)                                                    'initialize pBuild
   pBuild.RenderStream $Pin_Category_Preview, $MediaType_Video, pCap, Nothing, Nothing   'render the live source

   'pWindow is Global so that it can be used in %WM_Size
   pWindow             = pGraph
   pWindow.Visible     = %OATRUE
   pWindow.Owner       = hwnd
   pWindow.WindowStyle = %WS_Child Or %WS_ClipSiblings Or %WS_ClipChildren
End Sub

[Gary Beene]

Yippee! At 9:43pm Dallas time I successfully used DirectShow to display live video from my LogiTech C920 camera and then from my Elmo LX-1 camera.


The code is not pretty, it's not optimized, I have questions about why some of it works, I still have more studying to do, I may have done a few things incorrectly, I left out debugging code, I know a few things wrong with it, it has no features of any significance and I'm certainly not the first person to ever do it. But you can't imagine how happy I was when the video popped on the screen a little while ago. I've been wanting to do this for over 3 years now. I had not been able to talk myself into taking on the learning curve of both COM and DirectShow.

I had told myself earlier today, after spending almost 4 days working with DirectShow documentation and code, that barring any immediate success that I might have to stick with an undesirable alternative, which is a C# app that uses the AForge libraries to display a camera in a Window that my PowerBASIC app can control with a registered message.

Jose's Play-a-File COM example, plus some additional tips he gave me paved the way for getting this code to work, that plus poring over the web and going over the MSDN section on DirectShow and playing with translating the C++ examples until I was blue in the face.

Ok, it's not a monumental achievement, but I'm still a very happy camper!

I'll work up something better, but here's the code that I have. Give it a try, let me know how it works, and give me any suggestions/corrections that come to mind! There are all kinds of tests, changes, corrections, improvements, features and such to think about. But the needed first step was simply to demonstrate previewing live video.

'Compilable Example:
#Compile Exe
#Dim All
#Debug Error On
#Debug Display On
%Unicode = 1
#Include "Win32API.inc"
#Include Once "dshow.inc"
#Include Once "ole2utils.inc"   ' For IUnknown_Release

$Camera = "Logitech HD Pro Webcam C920"

%WM_GraphNotify = %WM_User+13

Global hDlg As Dword
Global pWindow As IVideoWindow            'Display Window

Function PBMain() As Long
   Dialog New Pixels, 0, "Direct Show - Video Preview",,,600,400, %WS_OverlappedWindow To hDlg
   Dialog Show Modal hDlg Call DlgProc
End Function

CallBack Function DlgProc() As Long
   Local rc As Rect
   Select Case Cb.Msg
      Case %WM_InitDialog
         DisplayVideo hDlg, $Camera
      Case %WM_Size
         GetClientRect hDlg, rc
         If IsObject(pWindow) Then
            pWindow.SetWindowPosition(rc.Left, rc.Top, rc.Right, rc.Bottom)
            RedrawWindow hDlg, rc, 0, %RDW_INVALIDATE Or %RDW_UPDATENOW
         End If
   End Select
End Function

Sub DisplayVideo (ByVal hwnd As Dword, DeviceName$)
   Local pGraph          As IGraphBuilder           'Filter Graph Manager
   Local pBuild          As ICaptureGraphBuilder2   'Capture Graph Builder
   Local pSysDevEnum     As ICreateDevEnum          'enumeration object
   Local pEnumCat        As IEnumMoniker
   Local pMoniker        As IMoniker                'contains information about other objects
   Local pCap            As IBaseFilter             'Video capture filter
   Local pControl        As IMediaControl
   Local pEvents         As IMediaEventEX
   Local hr              As Dword
   Local pwszDisplayName As WStringZ Ptr
   Local pceltFetched    As Dword
   Local pbc          As IBindCTX
   Local pPropBag        As IPropertyBag
   Local varName         As Variant


   'necessary objects
   pGraph      = NewCom ClsId $CLSID_FilterGraph          'filter graph
   pBuild      = NewCom ClsId $CLSID_CaptureGraphBuilder2 'capture graph builder
   pSysDevEnum = NewCom ClsId $CLSID_SystemDeviceEnum     'enumeration object

   pControl    = pGraph
   pEvents     = pGraph

   'get the Moniker whose friendly name matches DeviceName$
   hr = pSysDevEnum.CreateClassEnumerator($CLSID_VideoInputDeviceCategory, pEnumCat, 0)  'list of monikers for video sources
   Do   'iterate the list until the moniker friendly name matches the chosen device
      hr = pEnumCat.next(1, pMoniker, pceltFetched)
      If hr = %S_False Then Exit Do
      hr = pMoniker.GetDisplayName(pbc, Nothing, pwszDisplayName)                 'get complex camera name
      hr = pMoniker.BindToStorage(Nothing, Nothing, $IID_IPropertyBag, pPropBag)  'get info about Moniker
      If SUCCEEDED(hr) Then
         hr = pPropBag.Read("FriendlyName", varName, Nothing)   'get friendly name
         If Variant$$(varName) = DeviceName$ Then Exit Do       'this Moniker matches the one of interst
      End If
   Loop

   hr = pMoniker.BindToObject(Nothing, Nothing, $IID_IBaseFilter, pCap)                  'create device filter for the chosen device
   hr = pGraph.AddFilter(pCap,$Camera)                                                   'add chosen device filter to the filter graph
   hr = pBuild.SetFilterGraph(pGraph)                                                    'initialize pBuild
   pBuild.RenderStream $Pin_Category_Preview, $MediaType_Video, pCap, Nothing, Nothing   'render the live source

   'pWindow is Global so that it can be used in %WM_Size
   pWindow             = pGraph
   'pWindow.Visible     = %OATRUE    'seems to not be required
   pWindow.Owner       = hwnd
   pWindow.WindowStyle = %WS_Child Or %WS_ClipSiblings Or %WS_ClipChildren

   pControl.Run
End Sub

[Paul Purvis]

I spent 2 days on 4 or 5 lines of code this year.
I think I have you beat on characters coded per minute(hour) if lower is better.

[Dave Biggs]

Hi Gary,

Congratulations!

Tried your test app using the built-in Camera on my Surface Pro and it worked fine: Microsoft LifeCam Front (and Rear).

Good work

[Gary Beene]

Hi Dave!
Thanks! I'm glad to hear it worked on your end. I feel a gbVideo utility coming on, maybe today!

[Pierre Bellisle]

>"give me any suggestions/corrections that come to mind!"
Quickly, two minor adjustements:
GPF if no connected camera.
Use of a connected camera even if $Camera does not match.

Beside this...
You builded a plane that really fly!
Congratulations Gary! Good job :-)

[Gary Beene]

Howdy, Pierre!
Thanks - I'll cover those in the Source Code post that I hope to make tonight. I have a few more features I want to incorporate as well.