Craigslist Watcher Program

Posted by Mike Dwyer | Filed under

As of right now, this is just a teaser, but by the time anyone reads this, hopefully I will have the source code up. 

My new project is a craigslist watcher program.  The background is, anytime I want to search on craigslist for something (cheap), I have to refresh the page all day long.  Of course thousands of others could be waiting for the same item, so what happens most of the time is I find the item too late and lose the item.  I have found other watcher programs , but they did not do what I want (I am sure there are some that cost money).  The sample programs I found could either only search for one item, or only searched new items.  I wanted a program where I could put in a bunch of different search terms, and have the program first load what is currently there, then monitor the feed.  I also wanted to be able to remove items that I know I don't want, but not have them show back up in the list.

 I found some source code that was close to what I want.  It had a bunch of failures, such as only searching one item, using its own search algorythm instead of Craigslist, etc. In the end, I ended up modifying most of the code, and then adding all the functionality I wanted. I have to say, I am pretty impressed with myself on this one.  The program is running great thus far.  I have to do a little cleanup before I post the source, but should have that done in the next few days.  Currently it is in C# connecting to an Access DB.  I am going to convert Access to SQLite to make it a little more portable, especially to convert the app to run on Android. 

 To do: 
Run as a service
Text message
Notification on tabs
Convert to Android
Convert DB to SQLite
Email

 

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Motorola Droid First Take

Posted by Mike Dwyer | Filed under

I generally don't jump onto the new phone bandwagon, but my old phone, the Samsung i-760 was such a piece of garbage that I figured I would go with the Droid.  I do have to confess, I was pretty excited when I first heard and read about the Droid, so I am sure I was destined to switch anyways.  Let me begin by saying, I will not be comparing this to the the Iphone, you can find that elsewhere.  I will however compare some things to Windows Mobile, which is a silly comparison since Windows Mobile is a terrible operating system that looks just as clugy as it did when they had it on the old ipaq pdas. Really Microsoft, get with it. 

The Droid itself looks fantastic.  It is amazing how slim it is with a fold out keyboard.  I have found the keyboard is really nice.  I do like my old keyboard on the Samsun g a little better as far as how it feels when typing, however this new keyboard has a "@" and a "/" key which my old phone did not.  I really like having those keys.

With my old phone, if I was talking to someone, my phone screen would go into power saving mode which meant I had to turn the phone back on to disconnect.  This was really annoying when being on calls where you have to follow instructions and enter in keys.  The Droid, however, does go into power saving mode, but it detects when you move the phone away from you ear and the screen comes back on.  This is actually really nice.   Speaking of dimming, I have noticed the brightness of the screen changes quite frequently from somewhat dim to really bright.  I haven't figured out this problem yet, but will work on it.

 Listening to music has been much better as well.  On my old phone, the headphone jack was the smaller jack so I had to have a converted.  Also, when listening to music with headphones, when someone called me, the music stopped coming through the headphones and started playing through speakerphone, which was annoying when I was on the train.  The Droid, on the other hand, has a standard headphone jack size and I have not had problems listening to music, even when the phone rings.

One of the only problems I have had so far is with the Google Marketplace.  It seems the search is really bad.  For instance, I wanted to search for a Craigslist application, so I typed in a search term of "craigs".  I got 0 search items.  I had to type in  "craigslist", then a bunch of apps came up.  I tried this with a couple other searches and had the same problem.  I even tried going to the Google Apps website in my desktop browser.  From there I found some apps I wanted.  However, when I tried to find them via my phone I could not find them.  I am a little surprised that Google screwed up the marketplace search, but hopefully they will get that fixed.

I am still getting accustomed to everything.  I will be testing the internet connection after work.  My goal is to see if Pandora will play from the point I leave the office to the point that I get home.  My old phone could not do it for more than a minute without breaking up, and I am in downtown Chicago.  I have a feeling the Droid will do much better, but we will see.

  

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Prevent spam in blogengine

Posted by Mike Dwyer | Filed under

Ok, so having my blog has been nice, especially when I started receiving comments.  Little did I realize the comments were auto comments that was just spam so people can post their website info for advertising.  Well, after a while it got annoying trying to clean the comments.  What I realized is, in the comments section, people can post a website url, which is the reason for the advertising.  My first thought was I created a stored proc as follows:

create procedure dbo.spRemoveSpam as

delete from be_postcomment where isnull(Website,'')<>''

and isnull(Website,'') not like '%mypersonalwebsitehere%'

 I then ran that every night.  I could have created a trigger, but I just don't like them since I forget they exist and weird stuff happens!  I also could have just scheduled this to run all the time.  What I decided to do instead, was just add code into the file in the core dll file dbLogProvider.cs in the UpdateComments method, before the execute query, I changed the last line in that method:

 cmd.ExecuteNonQuery();

 TO

if (dpWebsite.Value == string.Empty) cmd.ExecuteNonQuery();

This will ensure that if someone fills in the website field, it just won't save.  I will eventually put text on the page as well stating that, but thus far the only people that post in the website field are spammers. 

 I suppose a CAPTCHA would be nice, and I am sure there is a module somewhere, I am just being lazy!

 

12/1/2009 UPDATE
I found a better way to handle this.  In the User Controls folder is a file called CommentsView.ascx.cs.  Do a search for:

if (website.Trim().Length > 0)

 {

if (!website.ToLowerInvariant().Contains("://"))

Right above the second IF statement, put a return.  This will stop the post altogether from moving forward and emailing you.  Final code looks like:

if (website.Trim().Length > 0)

 {

return;

if (!website.ToLowerInvariant().Contains("://"))

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Vibe Streamer vs Orb Streamer

Posted by Mike Dwyer | Filed under

I figured after all these years, surely there would be a lot of choices for those that want to stream music and videos from their home computer to their work computer or mobile device.  Well, turns out that I was wrong.  The background is this, though I really like Pandora and Last.FM, sometimes I just want to hear some of my own tunes.  I am one of those people that like to put all my music on one device to always have with me so I don't have to reconnect and remove and add music.  When I had my Zune, I put in the 80gb drive just so I could accomplish this.  Then I bought my crappy Samsung i-760 phone and realized I could put my music on there.  Well, that is nice except I only get 8gb of space on that tiny little card, that just didn't cut it. 

 That is when I switched to Pandora and Last.FM, however sometimes I want control over what I am listening too, or I want to hear a whole album, so that just didn't work.  The problem I had to solve was how to stream my music from home to work, when at work we block almost every port except the basics, so I need something that runs on port 80.  At this point, I found Vibe Streamer and Orb.  The biggest difference off the bat is Vibe Streamer only handles music, whereas Orb handles music, photos, videos. 

So, at first I started with Vibe Streamer.  It was easy to install, I put a mapping in my router to route traffice to my computer, and played music stored on my freenas server.  Worked great.  The interface was really good, easy to create on the fly playlists, etc. Everything plays as a flash file, so no need to worry about codecs.  The downside with Vibe is it does not work on mobile devices, and does not play videos.  The videos really doesn't bother me too much, though it would be nice to have.  The fact that it doesn't play on my windows mobile device though was kind of annoying.   Vibe does not run as a service, which to be honest is pretty stupid.  It does have a setting to load automatically on startup, but on 2 seperate computers that has not worked.  Plus, the app has to be loaded with an icon on the taskbar, would prefer it to be a background task.  There is a way through the windows resource kit to have it run as a service, I will be testing that later.  As far as I know, there is no iphone app to work with this, or any mobile device for that matter, at least until they have flash support.

After a hard drive crash and a system rebuild, I decided to give Orb a shot.  Orb, though runs on your local machines, has its hands in the orb.com servers.  No worries, your music and files stay on your own computer, I think they just keep a list for quicker management.  Once I installed the software and logged into my orb account, I was instantly pretty confused.  I just wanted to play my music.  Eventually I got it figured out, I think it was just running slow because it was still indexing my files.  The first thing to note is, on my system, everything runs in Windows Media Player instead of flash.  Personally, I don't like media player as it seems slow to buffer and play.  I have noticed a longer than expected pause between songs, and a lot of times when the song starts up it starts in almost a slow motion sound then picks up to regular speed.  The web interface I think is pretty clugy.  I have not figured out how to just quickly add songs to my list that I want to listen too, or randomize play.  More importantly, to me, is I can not see what is currently playing, unless I look at the window media player window.  We tested ORB on an iphone.  We can watch videos through the free iphone app, but you have to pay $10 to be able to listen to music.

I then tried to watch a video on my phone on the train.  Now, keep in mind, my Samsung i-760 is a piece of junk and has terrible reception, the video did not use the whole screen and was really choppy and was just continually trying to buffer.   Orb is nice here because it detects your connection rate and adjusts the video before playing it, but I think because my phone is a piece of junk that it just does not maintain a constant connection rate.  Would like to hear from others on the video on phones.

So, my overall opinion?  Well, I will be reinstalling Vibe Streamer for my music.  The interface was just that much better and easier to use.  When I switch to a google phone sometime early next year, I may try Orb again, or I may just have orb run on another port altogether, if that is even an option. 

Vibe streamer can be found at: www.vibestreamer.com
Orb can be found at: www.orb.com

Please let me know if you have found something better!

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Windows 7 Professional

Posted by Mike Dwyer | Filed under

So, recently I had a hard drive go out (thanks Seagate), which left me the option to either reinstall Vista, or just install Windows 7.  Well, not a real tough choice as any Vista user will tell you.  I figured Windows 7 had good reviews, and really, it just couldn't be worse than Vista.  I put in a fresh new drive, and let the install go.  Keep in mind, my machine is a quad 64 bit processor, so it is a little beefy, and I have 3 monitors hooked up (foreshadow...).  After 20 minutes of the windows 7 install, with me really having no interaction with the install, the computer rebooted and the screen was just black.  Personally I thought since it has only been 20 minutes, the install must just be getting ready to start.  However the screens stayed black, though on one of the screens I could see the mouse cursor.  I tried rebooting and the same thing.

Welp, I of course went to google (sorry Bing) and did a search.  It turns out that even in the beta version, people with multiple monitors connected had problems.  The solution?  Well, just disconnect all but one monitor.  Once I did that, the machine booted to Windows 7.  I couldn't believe it, the install was done in only 20 minutes.  About a half hour of windows updates, and all my monitors were connected back up and working.  To be honest, the monitor setup was so much easier than Vista.

 My personaly opinion on Windows 7 thus far is, it really is much better than Vista, though I have some getting used to certain things.

1. The task bar is nice.  My problem is I am accustomed to the task bar having all my apps spread out along the bottom for easy navigation for me.  Now they are all within one icon on the task bar.  This is nice to a point, but will take some time getting used to.  The cool thing about this though, is when you click on the icon, you can see previews of what is on the screen.  If you mouseover the preview, Windows 7 brings it to the foreground temporarily, and quickly for you to preview.  Amazing how fast this works.  The other nice thing is you can close windows from the listing as well.

2.  MSN Messenger kept loading, and I couldn't see any way to turn it off.  Turns out you have to press "alt" key to have the menu appear.  Not sure why Microsoft would do that as it is a little silly, but I guess that is what MS is known for.

3.  Same as Vista, but I could not edit and save the hosts file, unless I open up notepad in administrator mode (type notepad in the search box, right click it in the list, and run as administrator).

4.  The pop up boxes everytime I want to install something gets annoying.  Though, to be honest, I do not install stuff all too often.  But since this was a new build and I had to get all my software on there, it did get in the way.  No where near as annoying as Vista though.  There is a feature to make it so it doesn't grey out the entire screen when prompting you, which I of course did.  It would really be nice if they found a way to not ask me if I double click an application to run.  I think the assumption should be if I double click a setup file, then I really want to run it, no need to pop up asking me.

5.  All my apps loaded and ran just fine.  I have had no incompatibilities with drivers or programs (yet...).  This includes Visual Studio.net, sql server 2008, Adobe Photoshop CS, etc.

6.  File copying is MUCH faster than Vista.  I don't know how Microsoft screwed up file copying in Vista, but it seems to be fixed in Windows 7.  I haven't tried copying to a USB drive, which used to be dreadfully slow with my Vista machines, but am hoping that has been taken care of as well.

7.  The licensing is hard to understand.  From what I understand, you can upgrade from Vista to Windows 7, only if Vista is installed AND activated.  Now, the problem here is, my hard drive crashed.  Why would I want to reinstall the piece of junk Vista first instead of doing a fresh install?  Secondly, some people will have to call to have Vista activated again.  So, instead of a 20 minute install, I would have had to reinstall Vista (which takes a good hour), then call and speak to someone I can barely understand and read an insanely long key, which he can't understand, then he reads back a long key that I can't understand, then I would be able to install Windows 7.  Not a very smart approach.  Secondly, Windows XP users can not upgrade to Windows 7 from what I understand.  I think this is a poor policy, especially since the reason most people didn't upgrade was because Vista was a poor choice.  I think XP users should be able to buy the upgrade version, and do a fresh install with it. 

Though I think Windows 7 is a much better operating system than Vista, I still think Microsoft has a long way to go to understanind its users.  It may want to try to listen a little closer and treat them with respect, instead of force (i.e. forcing users to buy full version instead of upgrades).  But, as we have seen with Vista, Zune, Windows Mobile (which is really bad), Microsoft is definately not going to listen to users anytime soon!  I do recommend the upgrade to Windows 7 though, as I think most users will really like it, even XP users.

Thus far I have been very happy with Windows 7.  We'll see how it holds up over time now.

 

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Samsung I760 Review/Problems (don't buy this phone)

Posted by Mike Dwyer | Filed under

I cut out the long rant I had before for this blog, just because it was long winded.  Here is the short post:

1.  Terrible customer support, and I really mean terrible.
2.  Bad reception, not connector for external antenna.
3.  Battery life is short, thus the reason the phones comes with 2 batteries.
4.  Internet connection can not hold a fast connection, most often is connected at 1X.
5.  Did I mention the customer service?  Really bad, I think they hired a rock and made it a manager.

Currently rated 4.0 by 1 people

  • Currently 4/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Page execution order hints

Posted by Mike Dwyer | Filed under

We all have memorized the page life cycle of an ASP.NET page right?  Ok, so maybe not, unless we are in a job interview answering trivia/book questions by the interviewer.   So, I am not going to get into that as there are more than enough resources to help you there.  What I do want to address is some common areas where knowing the order of execution can be very important to avoid some common pitfalls from code I have worked with, as well as my own in the past.

We all know to begin our work in the page_load section of the page.  We know this is the starting point for most of us.  From there we can execute other procedures to build the page.  Now, first of all, do not put all your code in the page_load.  Not only does it get hard to follow at a high level what is happening during load, but you will find almost every time that you will have to move it, or start putting if statements around it making the page_load even harder to follow.  Why do I say that?  Because of the postback.

Ok, so you are following me here and you are telling me you use the isPostback to determine what to do if the user has made a postback.  You use the old:
 if (isPostBack) {
     executePostbackStuff();
     loadDropDowns2();

} else {
     executeNonPostbackStuff();

So far this is very simple.  If the page is a postback, I need to load a drop down list (from a database).  Now, let's say you add a linkbutton to your page, and you create an event for that linkbutton:
protected void linkbutton_Click(object sender, CommandEventArgs e) {
    removeItemFromDB(); //removes item that is used in dropdownlist

What's wrong with the above? Well, the page_load gets executed before the linkbutt_click event.  So, even though you remove the item from the DB that the dropdownlist is binding to, it will still be in the list until the next refresh.

The solution you say is the following: 

protected void linkbutton_Click(object sender, CommandEventArgs e) {
    removeItemFromDB(); //removes item that is used in dropdownlist
    loadDropDowns2();  //recreate dropdownlist

}

That is great, you have solved your problem in that you will just call the function to rebind the dropdownlist.  However, you have created a problem as well.  The problem is, that dropdown list will be bound twice on load.  Once in the page_load (from the postback), and a second time in the event.  Most times, you may not notice this, but if you are binding a large dataset to a repeater or gridview, your pages not only will load slow, you will be putting unnececsary work on the database server.

The solution in situations like this is to track which item caused the postback, and handle it accordingly, or follow below.

A second point I want to make here for beginners is, in your page_load you may have something like:

protected void Page_Load(object sender, EventArgs e) {
     loadDropDowns();

}

This of course will rebind the dropdownlist every time the page is loaded.  Keep in mind, you do not need to do this, and again only puts unneded overhead on the servers.  The first time the dropdownlist is bound, it will be stored in the viewstate and recreated on its own.  In case of postback, the page will still display the populated dropdown list.  So, this is what you will want for efficiency on your pages:

protected void Page_Load(object sender, EventArgs e) {
if (!isPostBack)   loadDropDowns();

}

This will work for any databound items that do not change between postbacks.  If some items do require a change, i.e. the linkbutton example above, then in the linkbutton event, call loadDropDowns(); which will then refresh the dropdownlist for you.

My advice for beginners and even intermediate developers, is to put breakpoints on all your data calls to see how many times they are called.   Each, on almost all occassions, should only be called once.  Try loading your page, go through all postback methods (i.e. form submit, linkbuttons, etc) and see if your breakpoints are hit multiple times.  If they are, you have some work to do.

 

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

custom Auto Complete Extender Extended

Posted by Mike Dwyer | Filed under

I have to be honest, I was a pretty big fan of the auto complete extender in the ajaxcontroltoolkit.  But, it got me thinking, which generally results in bad things happening.  I have some searches that search multiple fields in a database, for instance, different name fields (i.e. "To" and "From").  So, with the original extender, it would be difficult to bring the results since you really only had one column.  The result would be a one column list and the user would not know what field the result is coming from.  So, I thought it would be nice to bring back a column header.  Then I figured, while I am at it, to return a string delimited by "~~" which represents table columns and have the javascript build a table for me.  So, the result is the autocomplete will show multiple columns (much like a gridview) with column header, and the user can easily see the different results that match what they are typing.

 

Phew, that was pretty cool.  But the width of the autocomplete div was all over the place.  So, I had to add a property so you (the developer) can resize the div width. 

 Now we are really getting somewhere.  The next step was to navigate through the table structure with both keyboard and mouse.  With the keyboard, you can use up and down to go through.  When you select the row you want and hit enter, the autocomplete will fill in with the text from whatever column matches what you are typing.  In the example, you will see text to tell you to type "223".  You will see three rows displayed.  Cursor down and hit enter.  You will see the first column text will fill in.  Instead, type in "Micha" and cursor down one row where the first name column has Michael and hit Enter.  This type Michael will automatically fill in.  Originally, my header row got in the way and the user was able to cursor down and select the header row.  Well, that is just silly, so I added another property to let the extender know if there is a header row. 

Can it get any cooler?  It turns out that yes, it can get slightly cooler.  When the list drops down, use your mouse and click any of the columns and you will see the autocomplete fill in with the value from the column you clicked.

I built the sample project quickly, I believe the solution file is actually in the customAutoCompleteExtender folder.  The second project is website which contains the default.aspx sample page and the ajaxServies.asmx/cs files needed to return results.  You can easily have the result populated from a database as I did.

customAutoCompleteExtender.zip (967.79 kb)

Currently rated 4.3 by 3 people

  • Currently 4.333333/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Logitech Orbit C# and C++ webcam control

Posted by Mike Dwyer | Filed under

Really just posting this to get some info out there.  This is a project I would like to enhance as far as the detection portion.  The current state of the project is I have written a C# .NET app running with unmanaged dll to control a logitech web cam.  For those unfamiliar with the logitech orpit, it is a pan tilt zoom camara (PTZ).   The goal of this app was to have the .NET code control the webcam motors.  The second part is to add face detection.  I use the openCV for the face detection and tracking.  Currently the app works fairly well in my office following me, but not so well in the living room following others at a distance.  Part of the problem is the detection that I am using is very slow.  I haven't fully tied into the openCV, so that is part of the problem.  Secondly, the code I use to determine which way to move the motors of the cam needs a little work.  It tries to follow the motion of the face and move, but it is a little choppy.  If I move slow, it does in face follow me and zoom to keep me centered.  If I move to quick, it will lose me.  I have not coded in to follow the motion and make a guess as to which way to keep moving to try and find me again.  I am looking for others to help keep this project moving as I would like to add other detection capabilities besides just faces.  

 The code attached has a clever project name of "test".  In it you will find one of the projects has a form.  Open the form to see the different controls, I think it is all pretty self explanatory on the controls on the form.  The main point here was to run the unmanaged code to control the camara.  This should be very easy from the samples included to run in an ASP.NET webcam to monitor your house from remote and control the direction of the camara.  The face detection routines, like I mentioned, or a little slow, but can be pulled out for monitoring your house from remote.  Please upload any changes/ideas.  If you would like to help with this project, feel free to contact me.

 The orbitCamControlDotNet.zip references the project webcamlib.  I had all folders in c:\projects.  If you create your own folder structure, make sure to reference the project accordingly.

 Also, you will need to get and install OpenCV.  Make sure you download version 1.0 (2006-10-19 15:43), otherwise you will get compile errors.  You can download opencv from:  http://sourceforge.net/project/showfiles.php?group_id=22870&package_id=16937

NEW:  If you use a newer version of OpenCV, Open objectLocater.h
Look for: char* Helper( const CvSeq *seq, int index )
Add the word signed in front, so it will look like:
signed char* Helper( const CvSeq *seq, int index )

Rebuild and all should be good.

If you get an error that a module can not be found when running the app, go to the binl folder for openCV (c:\program files\opencv\bin) and copy the following files to your working bin folder:
cv100.dll (or cv110.dll for newer versions)
cxcore100.dll (or cxcore110.dll)
libguide40.dll

OrbitCamControlDotNet.zip (2.12 mb)

webcamlib.zip (76.33 kb)

Cascades.zip (364.13 kb) (unzip into a folder called cascades in the folder where the exe runs)

 For the record, the face detection code and explanation can be found at:
http://www.codeproject.com/KB/library/eyes.aspx

 10/14/2009 UPDATE:  Ben Ford has given me a version of PTZ that uses pure .NET (no external DLLS).  I have not looked at this yet, but thought I would post as is.  I assume it is only camara control and not face detection, so this may give most users a little more what they want:  ptz.zip (227.56 kb)

Currently rated 4.8 by 5 people

  • Currently 4.8/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

ajaxcontroltoolkit

Posted by Mike Dwyer | Filed under
I was pretty excited to see the ajaxcontroltoolkit and have tried to use it quite a bit.  Unfortunately, I have run into walls with real world implementations of these items.  Fortunately, the authors have made it open source so making changes, though difficult at times, is possible.  I have customized the autocomplete extender and am currently working on the masked edit extender.  I will post all code once I have updates.  I would be interested in others to work on some of these extenders, or at least post some wish list items as I make releases.  I will see as we go along, what is needed to post the items to codeplex so maybe they implement some of the customizations.

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5