I've seen a few people have problems with this data set when trying, for example, to find the average worldwide takings, since the values are text. I don't know much SQL, so don't know if there is a good solution for this. There seems to be a MONEY type, but I don't think it's implemented on KA (?).

petercollingridge
you mean that?:
SELECT AVG(CAST(SUBSTR(domestic_money, 2) AS REAL)) AS AVG_MONEY FROM top_movies;

@petercollingridge

AVG(domestic_money) doesn't work because of the $ in front. The fact that the data type is TEXT won't cause an error--look at AVG(year)--that works just fine.

That being said, AVG(SUBSTR(DOMESTIC_MONEY,2)) is all you need.

Even if a workaround works for the exercise, I don't want to encourage my new hires to learn that an optimistic substr is OK in a production system.

I find that the worldwide figure data for Avatar, Titanic and Furious 7 has some problems. The integer I get is always 2 for Avatar, please help.

Used Replace function to replace the comma, the substring function to start from the second position to omit the $ sign, and finally the convert function to convert to decimal data type.

select convert(substring(replace(worldwide,',',''),2),decimal(6,2)) as worldwide_gross from top_movies;

I noticed some carrots at the end of some of the year cells. What do those mean?

@abatt0ir: The source (http://www.boxofficemojo.com/alltime/world/) has notes that go with the chart. One of them explains the carot:
"^ Indicates the movie made its gross over multiple releases."

This is a really good example of data cleansing requirements for my students though they'd have almost no chance of understanding how to do this from the course content covered so far.

In order to return an average worldwide gross value figure (assuming in millions of $US) from the dataset provided that is correct as per data on 7Feb16 ($862.84) and formatted with a dollar sign at the front, you'd need to do the following.
SELECT '$' || ROUND(AVG(CAST(SUBSTR(REPLACE(worldwide,',',''),2) AS REAL)),2) AS AVG_WorldWide_Gross_in_Million_USD FROM top_movies WHERE CAST(SUBSTR(REPLACE(worldwide,',',''),2) AS REAL) != 0;

Thanks to all who have provided feedback in the comments above.

Is it better practice to just Find and replace in notepad all the $ or use the cast and all that ?

Why the first raw duplicates column names? Also it looks that starting with domestic_percent it doesn't match.

I was trying on khan academy to use this data for the SQL "Data dig" project but ran into a weird result after running this sql statement:

/* show how much money the Pirates series made */
select "All Pirates Series", SUM(CAST(replace(trim(worldwide,'$'),',','') as REAL))
as "$ Total WorldWide" from top_movies where title like 'Pirates%';

Instead of the expected value of 3729.6, the select returned 3729.6000000000003 . I cannot even guess why at this point. I tried using FLOA type as well in case it makes any diff. I even tried an extra trim around the replace statement in case there were extra spaces messing things up. What am I missing? (thanks in advance)

In order to play around with this data set on Khan Academy, use the following style of code rather than simply using the provided column name:
CAST(REPLACE(SUBSTR(worldwide,2),',','')AS real)

• Here I have taken the column 'worldwide' and started considering the values only from the second position using SUBSTR so that dollar signs are ignored.

• Then I have used REPLACE to basically replace the commas with blanks.

• Finally I have changed datatype from string to real using CAST.

Hope this helps!

Why is it not possible for me to use TOP keyword in the editor? I have found the keyword LIMIT to be working but TOP gives a syntax error :(

Since within the raw data, there are some years that end up with '^', which will cause duplicate value when remove the '^' by SUBSTR, here comes my question, if want to make a query to select those years that total yearly worldwide gross higher than some numbers such as 1000, how to do it. my solution seems can't remove the duplicated year.

SELECT DISTINCT REPLACE(year,'^','') AS Year, ROUND(SUM(REPLACE(SUBSTR(worldwide,2),',',''))) AS sum_world_gross FROM top_movies WHERE worldwide !='Worldwide' GROUP BY Year HAVING ROUND(SUM(REPLACE(SUBSTR(worldwide,2),',',''))) > 1000;

The result is weird, for example, there are 2 2008 with different sum_world_gross

In order to play around with this data set on Khan Academy, use the following style of code rather than simply using the provided column name:
CAST(REPLACE(SUBSTR(worldwide,2),',','')AS real)

• Here I have taken the column 'worldwide' and started considering the values only from the second position using SUBSTR so that dollar signs are ignored.
• Then I have used REPLACE to basically replace the commas with blanks.
• Finally I have changed datatype from string to real using CAST.

Hope this helps!

It did. Thanks. A shame this wasn't included in the actual course (so far anyway).

What's this CAST(REPLACE(SUBSTR())) about? Is this even covered in the tutorial? where do I put this.

Hi,
Is there some reason why all variables are "TEXT", when there was possibility of INTEGER or DECIMAL?

Thank you

Hi,
i have same problem but i find new solution:

SELECT * FROM top_movies;
SELECT studio, AVG(worldwide) as avg_worldwide FROM top_movies group by studio ;/*(bad response)*/
SELECT studio, AVG(replace(worldwide,'$','')) as avg_worldwide FROM top_movies group by studio ;/*(bad response, i check in excel: avg all valueBV=871,1)*/
SELECT studio, AVG(substr(worldwide, 2 , 10)) as avg_worldwide FROM top_movies group by studio;/*(bad response)*/
SELECT studio, AVG(substr(worldwide, 2 , 10)) as avg_worldwide FROM top_movies group by studio,worldwide;/*(i try group by studio and worldwide and i found inaccuracy of values from response)*/

UPDATE top_movies SET worldwide = replace(worldwide,',','');
UPDATE top_movies SET worldwide = replace(worldwide,'$','');

SELECT studio, AVG(worldwide) as avg_worldwide FROM top_movies group by studio ;/*heureka it's good response*/

Hope this helps!

Hi,
What I did was to copy all the data in a txt and replace all the ´$´ with ´´ and then used that info.

I love Khan Academy but they have a tendency to challenge you with tests that require advanced methods they don't teach; offering this table as an option for one of their projects really takes the cake. Also, none of the suggestions in this thread work for said project.

@aaronhhelton Thanks for the feedback! You're right, this dataset has a lot of value that needs cleaning. I'll look for a new data set that's more appropriate for the project level.
(And sorry I missed so much conversation on this gist - y'all came up with some great approaches to cleaning it)

Here are some additional data sets for the project:
Pokemon - https://gist.github.com/pamelafox/26b9357c6120d3d0d27c2b90c950e8c6
Superbowls - https://gist.github.com/pamelafox/3d8ab901def3577a8bfc881f6255d3f6
NBA players of the week - https://gist.github.com/pamelafox/6cda1115e2bcbc43e62cf4ff9d4158c7
NASA astronauts - https://gist.github.com/pamelafox/b98e03caca7d1ec14394a90ec1512cff

I have also updated this gist with cleaned numeric data.

Cool!

Good! It works fine.

alguien que hable español