Sunday, March 9, 2008

The Linux Flash Webcam Headache

Update: this has been turned into a dedicated web page: The Flashcam Project.

Let me start this blog with a subject that has been quite often addressed: "How to make my webcam work with Flash on Linux?"
Even if there is a driver for the hardware, it might be not recognized by Flash.
I will propose a solution, even if it is a work-around it will allow many users to consider Flash as a potential video conference solution with their favorite platform.


But first of all, it is necessary to describe the problem at large. I have no intent to argue or offense, just list facts.

Yes, there is a man behind Flash for Linux at Adobe, many should respect this first. Mike Melanson has been working hard to make it work especially when it comes to solve many interface issues (audio, video, fonts, window manager, network layers, ...). He did one very useful thing: through a dynamic library it is possible to hook a certain number of functionalities that allow to delegate some tasks like SSL and audio. This library is called libflashsupport and it is maintained here at SF.
Looking at the source gives hope when it comes to hook the video capture part, and hope vanishes when you notice that none of the functions are called by the plugin. Ok, it's disappointing, but the interface wasn't of a perfect design if you want to manage more than one video source. The GetFrame interface could have been kept though.

Anyway, Mike chose V4L to capture video frames. This has been controversial since V4L is now considered obsolete and V4L2 is the preferred approach for video on Linux.
Some drivers still respond to V4L and I could myself use a Creative webcam thanks to the almighty Gspca driver (what a great job Michel, Sylvie et al). But Michel is heading towards V4L2 and newer drivers are V4L2 only.
On a brand new PackardBell laptop, I installed a Fedora 8 and could very easily setup UVC driver to support the builtin webcam. On my Linux desktop (old Fedora 4) I have a PCTV Rave PCI card which is perfectly supported by the bttv driver.
But none of these two video frame grabbers are supported (just nothing for UVC and garbage for bttv). So the challenge is to make Flash believe that both video devices are V4L compatible. Re-writing drivers to be V4L compatible is everything but smart.

I found a very useful driver designed for debug purpose called Vloopback. It is V4L compatible, thus predicted to disappear. Actually, it is a solution, well almost. This nifty useful driver creates video devices pipes where you can write frames on an end and grab them on the other end. But many thought Flash could not understand vloopback interface, wrong! There is a small issue with vloopback and Flash: vloopback creates two devices where the first one is the input and the second one is the ouput. So when you install the default driver you get, say /dev/video1 and /dev/video2 and video1 is the input. Ah! Unfortunately Flash iterates over videoN devices and stops enumeration when open fails whatever error comes out. Too bad, /dev/video2 is never reached because it fails to open /dev/video1 (the input). What an easy patch! Swapping device registrations in the driver to get the input after the output. And tada Flash could 'see' the loopback. I just had to find an simple V4L2 capture program (why rewriting one where the V4L2 author just provides one, capture.c) and make it send frames to the loopback.
One additional issue! That would have been too easy otherwise. Flash only supports 'read' method and YUV420P pixel format. Not a big deal a few lines of code later and I got my simple flashcam program to forward V4L2 devices frames to the V4L loopback.
The driver has also been modified to block on write calls when no one is reading the output end, this makes the flashcam endless looping program just not stupidly eating up CPU cycles.

Enough blabla!!! Show us the solution dude! Ok, ok. For once let's make it simple:
  1. Grab the flashcam program and the modified vloopback driver.
  2. type make flashcam to build the frame forwarder.
  3. Build the vloopback driver: make (note you must have V4L and V4L2 support in your kernel config).
  4. Install the driver as root: make install
Now you have all binaries. As root start the vloopback driver:
# modprobe vloopback inminor=9

The inminor=9 option tells vloopback to create the input device on /dev/video9. The output device will be the next available one after /dev/video (i.e: /dev/video1).
Kernel message should display:

vloopback: Video4linux loopback driver v1.1.1 vloopback: Loopback 0 registered, input: video9,output: video1

You may have to change permissions on /dev/video1 and /dev/video9 so users can access them.
As normal user now run flashcam program:

[olivier] $ ./flashcam -d /dev/video0 -l /dev/video9

Note: type flashcam -h for help about the utility.

By default flashcam grabs frames from /dev/video (with -d option you can specify where is your V4L2 device), it is told to forward them to /dev/video9 with the arguments shown above. From now on, /dev/video1 is a V4L compatible interface that grabs frames from /dev/video which is a V4L2 interface.

Fire up Firefox and get a SWF to test, why not this one (click Allow in the pop up otherwise the test will be totally useless)? Note: in any active SWF movie you can change the default video source with the contextual menu, right-click and select "Settings".
The first video source entry is the V4L2 device, which of course doesn't show anything, but the second one, which is the loopback...



tada!


This successfully worked with Bttv driver and UVC driver.

The solution isn't packaged at all as you can see. And the target audience here is Linux developers. There are certainly many things to improve, flashcam could support many more pixel formats than the default used one, automatically searches the loop device, etc.
It might become an easier to use stuff if this found useful by the community. This is the beginning, feedback is appreciated.


Keep in mind that this is a kludge until there is a true support for V4L2 in Flash.

Enjoy.

-Olivier

35 comments:

Ethereal Panda said...

I stumbled across this while setting up my webcam, and I have to say excellent work. I was able to get this up and running quite quickly.

When I discovered that v4l2 wasn't supported by adobe's flash 9, the first thing I wondered was why someone hadn't developed a simple translator like this. This will work until the root of the problem is addressed. Thanks a bunch!

M said...

Olivier,

I'm getting closer, but still had some problems.

Everything compiles without error, but I'm not seeing the loopback device?

# make
make -C /lib/modules/2.6.21.7/build SUBDIRS=/root/flashfix/vloopback-1.1.1 modules
make[1]: Entering directory `/usr/src/linux-2.6.21.7'
Building modules, stage 2.
MODPOST 1 modules
make[1]: Leaving directory `/usr/src/linux-2.6.21.7'
# make install
install -d /lib/modules/2.6.21.7/kernel/drivers/misc
install -m 644 -c vloopback.ko /lib/modules/2.6.21.7/kernel/drivers/misc
/sbin/depmod -a
# /sbin/modprobe videodev

# /sbin/modprobe vloopback inminor=9

# ls -ltr /dev/video*
crw------- 1 root root 81, 1 2002-08-30 19:31 /dev/video1
crw------- 1 root root 81, 0 2002-08-30 19:31 /dev/video0
lrwxrwxrwx 1 root root 6 2007-10-14 20:19 /dev/video -> video0
# lsmod|grep v4l
v4l2_common 16896 3 usbvision,bttv,videodev
v4l1_compat 14596 2 bttv,videodev
#

I've tried rebooting and it didn't help?

Any ideas?

Thanks for your help.
Michael

Olivier "Swift" Debon said...

Michael,

I don't see what problem you're talking about. Where I see no errors. You don't have to load videodev module, it was already there since you have bttv loaded.
vloopback didn't report any error. So everything is ok. Why don't you start flashcam?

-Olivier

M said...

# ./flashcam -l /dev/video9
bash: ./flashcam: No such file or directory
# dmesg|grep video
Boot video device is 0000:00:02.0
Linux video capture interface: v2.00
unionfs: do delay copyup of "videodev.ko"
unionfs: do delay copyup of "video-buf.ko"
bttv0: registered device video0
USBVision[0]: registered USBVision Video device /dev/video1 [v4l2]
vloopback: Loopback 0 registered, input: video9,output: video2
#

# ls -ltr /dev/video*
crw------- 1 root root 81, 1 2002-08-30 19:31 /dev/video1
crw------- 1 root root 81, 0 2002-08-30 19:31 /dev/video0
lrwxrwxrwx 1 root root 6 2007-10-14 20:19 /dev/video -> video0
#

I still only see the video devices that existed before I started loading your fix?

M said...

Opps... This is what I meant to post in the top of the last message...

# ./flashcam -l /dev/video9
Failed to open loop device: /dev/video9 No such file or directory
#

Olivier "Swift" Debon said...

You might have to create the devices manually.
# MAKEDEV /dev/video2
# MAKEDEV /dev/video9
Of course thoses devices MUST be the ones reported by vloopback.

Anonymous said...

I haven't tested this yet, but I looked at the code and got concerned about audio.

The capture code, flashcam.c, only seems to deal with video.
How is audio handled?

What would be required to get audio working? And what about audio-video synchronization problems?

As a side-note, I also have an UVC webcam, but never found any application capable of recording synchronized audio and video, while at the same time displaying the image being recorded.
It's too bad luvc-view doesn't support audio-video recording.

Anonymous said...

HELP!!!

Before any vloopback and flashcam I have:
crw-rw---- 1 root video 81, 0 2008-03-23 20:50 /dev/video0
crw-rw---- 1 root video 81, 1 2008-03-24 03:20 /dev/video1

video0 is my tv card and video1 is my Logitech Quickcam Fusion.

#modprobe vloopback inminor=9
#ls -l /dev/video*
crw-rw---- 1 root video 81, 0 2008-03-23 20:50 /dev/video0
crw-rw---- 1 root video 81, 1 2008-03-24 03:33 /dev/video1
crw-rw---- 1 root video 81, 2 2008-03-24 03:34 /dev/video2
crw-rw---- 1 root video 81, 9 2008-03-24 03:34 /dev/video9

#dmesg
usb 7-2: new high speed USB device using ehci_hcd and address 5
usb 7-2: configuration #1 chosen from 1 choice
uvcvideo: Found UVC 1.00 device [unnamed] (046d:08ca)
vloopback: Video4linux loopback driver v1.1.1
vloopback: Loopback 0 registered, input: video9,output: video2

I start flashcam as user:
$./flashcam -d /dev/video1 -l /dev/video9
Input device: /dev/video1
Size = 160 x 120

I remind that /dev/video1 is my webcam.

I try to use your flash application to test the webcam and the list of devices there is the following:
- none
- BT878 video (my tv card)
- UVC camera
- 2, undefined,

I select the last one, which according to you should say "Video loopback 0 output" and this is what I get:

#dmesg
ioctl32(operapluginwrap:7965): Unknown cmd fd(4) cmd(803c7601){00} arg(ffa66b68) on /dev/video2
ioctl32(operapluginwrap:7965): Unknown cmd fd(4) cmd(800e7606){00} arg(ffa66ba6) on /dev/video2
ioctl32(operapluginwrap:7965): Unknown cmd fd(4) cmd(80207609){00} arg(ffa6699c) on /dev/video2
ioctl32(operapluginwrap:7965): Unknown cmd fd(4) cmd(803c7601){00} arg(ffa67b78) on /dev/video2
ioctl32(operapluginwrap:7965): Unknown cmd fd(4) cmd(800e7606){00} arg(ffa67bb6) on /dev/video2
ioctl32(operapluginwrap:7965): Unknown cmd fd(4) cmd(80207609){00} arg(ffa67abc) on /dev/video2
ioctl32(firefox-bin:8279): Unknown cmd fd(60) cmd(803c7601){00} arg(ff860fe8) on /dev/video2
ioctl32(firefox-bin:8279): Unknown cmd fd(60) cmd(800e7606){00} arg(ff861026) on /dev/video2
ioctl32(firefox-bin:8279): Unknown cmd fd(60) cmd(80207609){00} arg(ff860e1c) on /dev/video2
ioctl32(firefox-bin:8279): Unknown cmd fd(61) cmd(803c7601){00} arg(ff862128) on /dev/video2
ioctl32(firefox-bin:8279): Unknown cmd fd(61) cmd(800e7606){00} arg(ff862166) on /dev/video2
ioctl32(firefox-bin:8279): Unknown cmd fd(61) cmd(80207609){00} arg(ff86206c) on /dev/video2

As you can see, I tried 2 different browsers and they both fail.

My linux system:
Linux stardust 2.6.21-2-amd64 #1 SMP


Thank you for any help you may give.

Olivier "Swift" Debon said...

64bit is not (well?) supported by Flash, not I'm aware of. All your commands were just perfect. I'll have a look at the ioctl32 errors.

Anonymous said...

Yes, I know 64 bits is not supported by Flash.
That is why I use a 32bits chroot environment to run both firefox and opera... and flash works fine in those.

I have now tried to compile and run flashcam in the chrooted system and this is what I get:

$./flashcam -d /dev/video1 -l /dev/video9
ioctl VIDIOCSPICT
Error[Invalid argument]

#dmesg
ioctl32(flashcam:11455): Unknown cmd fd(3) cmd(400e7607){00} arg(ffcc2b3a) on /dev/video9

I no longer know if flashcam should run in chroot or not. But results seemed to be better in 64bits.
In 32bits chroot, flashcam halts at open_loop()

If it matters, I can run lucvview either in jpg or yuv capture modes.

And I checked Debian site http://merkel.debian.org/~jurij/ for the kernel config and it says the kernel supports V4L1, V4L1_COMPAT and V4L2.


I'm out of ideas... :(

Thank you for taking a look at it Debon.

Anonymous said...

I have now tried using vloopback and flashcam for another purpose... to see if I can understand the error better.
The application is gyachi, a yahoo client with webcam support.

This is what I get from gyachi when using flashcam_64bits:
"An error occured at 'ioctl VIDIOCGCAP'.
Wrong device."
And another one for VIDIOCSPICT.

I'm unable to run flashcam in 32 bits... I get the same error stated in my previous post.

Swift said...

I don't know about gyachi, flashcam is Flash specific (opens a V4L2 device and forward frames to V4L1 one, with specific capture settings).
Running things in 32bit or 64bit environment creates more difficulties to figure out what's wrong. What I can see from the kernel messages is that ioctls are not understood which is likely to be a mistranslation from 32bit to 64bit (native) ioctls. Do you have compat_ioctl32 loaded? That might solve some issues.

M said...

I'm getting even closer...
# mknod /dev/video2 c 81 2
# mknod /dev/video9 c 81 9
# chmod 666 /dev/video2
# chmod 666 /dev/video9
# ls -ltr /dev/video*
crw------- 1 root root 81, 1 2002-08-30 19:31 /dev/video1
crw------- 1 root root 81, 0 2002-08-30 19:31 /dev/video0
lrwxrwxrwx 1 root root 6 2007-10-14 20:19 /dev/video -> video0
crw-rw-rw- 1 root root 81, 2 2008-03-24 08:42 /dev/video2
crw-rw-rw- 1 root root 81, 9 2008-03-24 08:43 /dev/video9
# ./flashcam -l /dev/video9
Input device: /dev/video
Size = 160 x 120

I see the "Video loopback 0 output" device in the flash selection and by selecting it, I get a blue screen. A blue screen usualy means the device is working but no video source is connected, but there is a working vodeo source so I assume the input is not correct for the capture device. I change it through "v4lctl -c /dev/video1 set input "Composite Video Input".

This does not help.

I'm also confused as to why it seems to use /dev/video which is a symlink to video0, which is my PCI capture card. I think I'll try all of this with a system that does not have any other video devices next.

Anonymous said...

@ Swift

# lsmod | grep compat_ioctl32
compat_ioctl32 13056 2 uvcvideo,bttv
v4l2_common 23424 6 uvcvideo,tuner,msp3400,bttv,compat_ioctl32,videodev

So in your opinion the problem is with the vloopback and not with flashcapture?


@ M
Have you tried this instead:
$./flashcam -d /dev/video1 -l /dev/video9

Swift said...

M,

You're done! Just set the composite source way before you start flashcam and even you start any Flash movie. I got the same thing with bttv. I use tvtime to set composite then I load the module, start flashcam and you'll get what I got.
Flash doesn't support any change during capture, it can even crash.

Anonymous,

I don't know actually, I see no issues in the flashcam proggy which is terribly simple. May be the driver should be written with 64bits support, I really don't know.

-Olivier

Swift said...

....
anonymous, I think the vloopback driver might need to explicitely declare a dependence on compat_ioctl32.
Please email me directly instead ofposting here. We'll find a way.

Swift said...

Something new is underway a more packaged thingie. A new article should be posted soon and a dedicated web page for the whole project.

64bit issue has been solved for anonymous.

I have removed some comments, if you post here and expect some help, you might answer the questions I ask, if you don't, well I'll dump the posts. Sorry.

-Swift

Swift said...

As stated at the very beginning of the article, a dedicated page now exists.
Check out Flashcam Project

edric.Yip said...

Hi Swift,

thanks for the great solution... i cover the last loophole in my linux. now i can forget about going into windows.

Anyway, as u mention in your article, it is possible to support better pixel. i am just wondering if there is anything i could tweak for that :)

my webcam UVC driver max at 320X240 ... currect loopdriver is at 160X120.

Swift said...

Hey,

I meant to choose 160x120, my UVC webcam sends frames at 640x480, but 160x120 is default size in Flash. You can change this with flashcam -s 320x240, make sure Flash isn't running it is likely to crash.

-Swift

m1c4a1 said...
This comment has been removed by the author.
Mike said...

Hi Swift,
this solution is nice. I get everything to work together but sometimes when I use flashcam -d /dev/video0 -l /dev/video9 I get select timeout.

This is a message from the original capture.h, but I am not sure what it means. Unfortunately nothing is to be found in the logs.

I am using a Logitech QC Pro 9000.

Perhaps you have an idea?

Regards.

Mike

Swift said...

Hi Mike,

Please get to the newest webpage, this has been fixed in The Flashcam Project.

-Olivier (Swift)

Qthrul said...

Worked like a charm. :)

Good job!

baphomet said...

to whom it may concern,

you guys are fucking awesome. works perfectly.

the professional amateur said...

Anyone interested in helping to convince Adobe to fix this issue in the Flash Player itself by implementing complete Video4Linux2 support, please visit Adobe's own Flash Player Bug and Issue management system, register an account, and vote for this Feature Request:

http://bugs.adobe.com/jira/browse/FP-204

Steph Technofeliz said...

Hi all...

Just information on the new Flash plugin v10 beta release and this Webcam bug... I've just tested it on my EeePC... and I have bad news : the V4L2 bug is not corrected in v10... It's a shame... :(
In fact the bug is still in "open" status in the Adobe DB Database

I hope that a good non-official Flash plugin will pop up now that Adobe has opened (sort of) the Flash specifications

Are you personaly working on such a plugin ? :)

Swift said...

Yes I know FP10 has no fix regarding this issue. Someone has posted a link to Adobe's KB so I suggest people register and vote (I did that myself I'm no workaround addict).

I use to have worked on a plugin a very very long time ago. The original GPL FLash Plugin was released in 1998 :) Then came the official one. To be honest I see NO reason to create an alternative like Gnash, this is where Linux community is wrong. Programmers should really not waste their time in this, this is almost discouraging for companies that want to support Linux (with some inaccuracies sometimes :)). Certainly a subject for a good rant. ;)

-Olivier (Swift)

dokeoslead said...

I partially agree with the "Gnash is wrong" idea, but at the same time I think it does provide some kind of incentive for Adobe (in this particular case) to keep working on Linux compatibility.

I've been trying myself to get Adobe to move its ass about the V4L2 problem by sending suggestions that they quote a price for the development of this feature and we gather community money to finance that (just in case that would motivate them) and getting several of my friends to report it as a feature request, but never got any answer. It's really one of these damn little things that are holding down Linux on the desktop...

So if Adobe continues ignoring the V4L2 problem, I see no other solution than

Yannick

dokeoslead said...

...than supporting the GNash movement (sorry about the partial message)

Patrick Balleux said...

Hi, Just to let you know that Adobe as release a RC of Flash 10 and it works quite well with V4L2 webcam.

I am currently using it on an AMD64 and a UVC webcam under Ubuntu Hardy.

It is a little tricky to install because there are some 32 bits dependencies that can be resolved quickly with "getlibs".

Swift said...

Merci Patrick.

Indeed latest version of Flash Player from adobe labs just works beautifully (10.0.0.569).

The project will still remain useful for embedded devices like N800 and EeePC.

-Swift

Disruptor said...

So i installed Flashcam and on the first execution of the program it allowed me to use the command:

> flashcam -d /dev/video0 -l /dev/video9

Which turned my cam on and showed the following output:

Size = 160 x 120
Pixel format = YUYV
Bpl = 320

But when i attempted to test the webcam in flash i still only got one option for my camera which was "CNF7129" so i selected it and hit allow but no picture appeared. I figured possibly with a new install it would require a restart so i restarted and upon loading my OS:Fedora 9 (x86), i noticed /dev/video9 was gone so i tried to run:

> modprobe vloopback inminor=9

But it did not add /dev/video9. I am still able to run flashcam without any parameters and receive the same otput except it says it is forwarding from /dev/video0 to /dev/video2 instead. There is still only one option in flash and video does not work with it selected.

Any Idea???

Thanks

Swift said...

I suggest you upgrade to latest official release of Flash from Adobe.
All this article and FlashCam Project are now to be considered obsolete for desktop use.

Disruptor said...

Thanks! I was totally not expecting Flash 10 to be out for linux/firefox yet good call. I upgraded and it works like a charm without flashcam, I would just like to say though good work on this program :-)