Your opinion on objective photo ratings

[kalendae]

Hi All,

I've always found the DPS community to be insightful so I am looking for some opinions on something kind of geekier than usual.

I have been working on a flickr app (EloGrade Objective Photo Ratings on Flickr - Application Sharing!) to calculate objective ratings for photos. The idea is to do comparisons instead of direct ratings and use statistics and a system called elo rating system.

Right now if someone asks you how good of a photographer you are, I feel like the only threshold is if you can say well I make a living off of it, or I made Flickr explore or I've been published, which are all not that measurable really.

I'd like to get your general opinions and suggestions and

1) does the whole thing make any sense?
2) would u find this useful? and would be great to hear reasons either way


Thanks

- Kal

[Nicole]

1. Sure it makes sense enough, I mean, essentially you can quantify anything if you really, really want to.

2. No, I wouldn't really find it useful because part of what I consider a good photo is something I like. Now, what if people generally favour photos of people over flowers or vice versa and therefore they tend to rank them higher? I wind up presented with a number of photos that I don't like and it doesn't tell me anything about how good the photographer is because lots of people might consider a photo of a scantily clad 20-year old better than a photo of a 90 year old simply because of the subject. I guess what I'm trying to say, very unclearly I might add, is that I find photography too subjective to apply such a subjective measure.**

That being said, I'll probably have a look at the site and play around with it a bit because the concept is interesting. I'll be interested to see if the higher rated photos are ones I prefer or not. But it's probably not something that I'll use more than once or twice to entertain myself.

** However, I'm also a qualitative researcher at the moment, so my opinion may be biased

[LeeR]

Thank you so much for posting this; it is a very interesting experiment. Even if it fails in its attempt to quantify the value of images, it is certain to raise new questions about why we value certain images over others.
To be honest, I found some comparisons very difficult. As often as I could I let my background in the arts inform my decision, but often that wasn't sufficient. Is an image of a baby more important that one of a flower? Is an old black and white of a train more important than a horse's mane? Sometimes yes, sometimes no. Either way, I found the exercise fascinating and look forward to the conclusions that are drawn from it.

[wulf]

Potentially it provides lots of fascinating information but not necessarily a good judgement on the merits of a photo. As LeeR notes, how can you properly compare a flower and a baby. Or, given two baby pictures, what if one was intended to please the family and succeeded in presenting the child as healthy and beautiful, while the other was related to a story about poverty and showed a suffering child in desperate circumstances? Assuming both are technically competent, how would the ratings fall?

I can see that it would provide information on things such as whether people prefer images of health and happiness to disease and sadness and perhaps even whether factors such as saturation and contrast can affect the popularity of an image. However, I'm still not convinced that it will reach the stated aim of reliably saying this photo is better than that one.

Popular != Better.

Wulf

[Photosbykev]

I don't think I would say anything if my images got onto Flickr Explore, have you seen some of the dross that makes it onto there???

[BCampbell]

It sounds like this is really a prediction tool, not necessarily a rating tool. The two can be linked, but it's an important distinction.

Anyway, I'm kind of with Nicole; this wouldn't really help me because I'm pretty well convinced (based on what I see on flickr, blogs, even here) that what I like isn't representative of what most people like. So on the consumer end, it won't do me any good.

I also don't have a goal of appealing to the widest audience possible -- otherwise known as the lowest common denominator -- so on the production end it doesn't help me too much either. I don't really care what the overall consensus on any of my photos is, I care about the input from more a more specific audience.

[jdepould]

Technically it's a crap image, but it's a really awesome crap image.

[LeeR]

Since I posted my first reaction to this message I have spent some time on the site and even posted some images. I have to admit, it is fascinating to see it at work. For the most part I have to agree with outcomes; the images with higher ratings do in fact tend to be better images, and, no, babies and kittens do not predominate.
I am not willing to go so far as to say this is a predictor of great images; as a matter of fact, I'm not even sure exactly what this is measuring. According to the developers it is measuring the consensus opinion of different images and making predictions about how someone who had never seen the image before would evaluate it.
For my own part, I have been very careful about which images I posted. That is, I figured that simple images with highly graphic compositions would do best and for the most part that has been true. My highest rated image (1987) is VERY simple and VERY graphic. ( I am using graphic in the artictic sense here; as in simple recognizable shapes and patterns.)
In observing other images that do well, I have noticed that good lighting and sharp focus will push you toward the top, but great atmosphere, like mist of fog, does even better. Scenes with people in them do better than those without, and color is better than b&w. No surprises there. What does all this mean? I have no idea, but I find it intriguing and plan to watch it for a while longer. This kind of rating has the potential of playing a significant role in the future. Advertisers spend billions every year on images and having a tool like this to predict people's reaction is likely to prove irresistable to those writing the checks.

[ZannieGirl49]

Originally Posted by LeeR

In observing other images that do well, I have noticed that good lighting and sharp focus will push you toward the top, but great atmosphere, like mist of fog, does even better. Scenes with people in them do better than those without, and color is better than b&w. No surprises there. What does all this mean? I have no idea, but I find it intriguing and plan to watch it for a while longer. This kind of rating has the potential of playing a significant role in the future. Advertisers spend billions every year on images and having a tool like this to predict people's reaction is likely to prove irresistable to those writing the checks.

I've posted a few images after reading your post because I was intrigued by the trends that you noticed. I purposely varied them to see how the ratings differed. I'm not the best photog but it's interesting to see how my image with people is doing better than the one without and so on. I'm going to continue to watch too, I just need to control myself. For me the comparing can get a bit addictive. I really enjoy looking at images created by other people.

I think you're spot on in terms of the potential for an app like this. What struck me is that I tend to make very quick decisions which I realize are much like those that I would make when seeing an image in a magazine. I'm curious about how other people compare. Do they take time to really look at the technical quality or is it more of a gut reaction? All in all, I find it interesting and will continue to visit.

[kalendae]

First of all, thanks guys for checking it out. Secondly, wow, when I said DPS was insightful, that was an understatement. Your thoughts have really made me consider the ramifications of the rating that I hadn't considered til now.

I think as a result I will first add something to the app to explain what it is not. Since without context the world press winning vietnam execution photo above would definitely rate lower than most photos. Thats a good point about what the rating does not do.

LeeR: really appreciate your comments especially. The photo/ratings data will be publically available for analysis. I've done some simple data mining on just easy EXIF data but being able to do those things you are talking about would be really great. hopefully that can be done at some point when theres more data or someone able to analyze the data that way.