Saturday 6 August 2016

Complete Coursera Big Data Specialization - University of California, San Diego

The University of California, San Diego's Big Data Specialization was developed alongside Splunk. It is another new upstart specialization which got underway this Fall, and focuses mainly on what first comes to mind when you think Big Data: the Hadoop/Spark ecosystem. It does, however, have some other topics thrown in as well, including hot topics such as graph analytics and machine learning.


The specialization contains the following courses:

▪  Introduction to Big Data
▪  Hadoop Platform and Application Framework
▪  Introduction to Big Data Analytics
▪  Machine Learning With Big Data
▪  Introduction to Graph Analytics
▪  Big Data - Capstone Project

Specialization coordinator Natasha Balac was kind enough to provide some further insight for us, answering the following questions.
























▪  Introduction to Big Data





















▪  Hadoop Platform and Application Framework




















▪  Introduction to Big Data Analytics





















▪  Machine Learning With Big Data



















▪  Introduction to Graph Analytics





















▪  Big Data - Capstone Project


























Tuesday 8 March 2016

Machine Learning Jupyter Notebook - Keyboard Shortcut (Command Mode & Edit Mode)



















Friday 20 November 2015

五种有效的学习方法 - 方法比努力重要


















1 目标学习法
掌握目标学习法是美国心理学家布卢姆所倡导的。布卢姆认为只要有最佳的教学,给学生以足够的时间,多数学习者都能取得优良的学习成绩。
教 学内容是由许多知识点构成,由点形成线,由线完成相对独立的知识体系,构成彼此联系的知识网。因此明确目标,就要在上新课时了解本课知识点在知识网中的 位置,在复习时着重从宏观中把握微观,注重知识点的联系。另外,要明确知识点的难易程度,应该掌握的层次要求,即识记、理解、应用、分析、综合、评价等不 同层次,最重要的就是明确学习重要目标,即知识重点。有了目标能增强我们学习的注意力与学习动机,即为了这目标我必须好好学习。
可见,明确学习目标是目标学习法的先决条件。目标学习法的核心问题,是必须形成自我测验、自我矫正,自我补救的自我约束习惯。对应教学目标编制形成性检测题,对自己进行检测,并及时地反馈评价,及时矫正和补救。

学 习目标与人生目标不同,它比较具体,可以在短时间内实现。它可以使我们比较容易地享受成功的欢乐。增加我们的信心。因此,目标学习法也是成功教育的主要 策略之一,同时,实现学习目标也是实现人生目标的开始,只有使大小、远近目标有机的结合,才会避免一些无效劳动的发生。


2 问题学习法
带着问题去看书,有利于集中注意力,目的明确,这既是有意学习的要求,也是发现学习的必要条件。心理学家把注意分为无意注意与有意注意两种。有意注意要求 预先有自觉的目的,必要时需经过意志努力,主动地对一定的事物发生注意。它表明人的心理活动的主体性和积极性。问题学习法就是强调有意注意有关解决问题的 信息,使学习有了明确的指向性,从而提高学习效率。

问题学习法要求我们看书前,首先去看一下课文后的思考题,一边看书一边思考;同时,它还要求我们在预习时去寻找问题,以便在听课时在老师讲解该问题时集中注意力听讲;最后,在练习时努力地去解决一个个问题,不要被问题吓倒,解决问题的过程就是你进步的过程。


3 矛盾学习法
矛盾的观点是我们采用对比学习法的哲学依据因为我们要进行对比,首先要看对比双方是否具有相似、相近、或相对的属性,这就是可比性。对比法的最大优点在 于:(1)对比记忆可以减轻我们记忆负担,相同的时间内可识记更多的内容。(2)对比学习有利于区别易混淆的概念、原理,加深对知识的理解。(3)对比学 习要求我们把知识按不同的特点进行归类,形成容易检索的程序知识,有利于知识的再现与提取,也有利于知识的灵活运用。

综观中学课本,可比 知识比比皆是,如政治内容中,权利与义务、民主与法制、物质与意识、和平与发展等等;如语文学习中,复句与单句、设问与反问、比喻与借 代、记叙与议论、实词与虚词等等;如数学学习中,小数与分数、指数与对数、奇函数与偶函数、平行与垂直等等;如化学学习中,金属与非金属、晶体与非晶体、 化合与分解、氧化与还原、酸与盐等等。对比学习法不仅可以用于同一学科内的学习,还可以进行跨学科比较,如学习政治可用语文中的句子分析法来分析政治概 念,如在学习近现代史中的民族解放运动时,又可以利用政治有关民族的基本观点,学习自然学时,可回忆一下有关语文课本中的有关科学家的传记文章,也可结合 唯物辩证法的有关原理进行学习。


4 联系学习法
唯物辩证法认为世界上任何事物都是同周围的事物存在着相互影响、相互制约的关系。科学知识是对客观事物的正确反映,因此,知识之间同样存在着普遍的联系,我们把联系的观点运用到学习当中,会有助于对科学知识的理解,会起到事半功倍的效果。

根 据心理学迁移理论,知识的相似性有利于迁移的产生,迁移是一种联系的表现,而联系学习法的实质不能理解为仅仅只是一种迁移。迁移从某种意义上说是自发 的,而运用联系学习法的学习是自觉的,是发挥主观能动性的充分体现,它以坚信知识点必然存在联系为首要前提,从而有目的地去回忆、检索大脑中的信息,寻找 出它们间的内在联系。当然,原来对知识掌握的广度与深度直接影响到建立知识间联系的数量多少,但我们可以通过辩证思维,通过翻书、查阅、甚至是新的学习, 去构建新的知识联系,并使之贮存在我们的大脑之中,使知识网日益扩大。这一点是迁移所不能做到的。

学习新知识就要想到旧知识,想到自己亲 身经历过的事,不能迷信权威,克服定势思维。把抽象的知识具体化,发挥右大脑的作用。如辛亥革命发生在1911年, 二次革命发生在1913年,护国战争发生在1915年,护法战争发生在1917年,这四个历史事件依次间隔二年,只要记住这两个历史事件的逻辑顺序,知道 其中任何一个事件的年代,就可以联想,推算出其它三个事件的年代。这是联想记忆法。

读书之法,既先识得他外面一个皮壳,又须识得他里面骨髓方好。——朱熹


5 归纳学习法
所谓归纳学习法是通过归纳思维,形成对知识的特点、中心、性质的识记、理解与运用。当然,作为一种学习方法来说,归纳学习法崇尚归纳思维,但它不等同于归纳思维本身,同时它还要以分析为前提。

可见,归纳学习法指的是要善于去归纳事物的特点、性质,把握句子、段落的精神实质,同时,以归纳为基础,搜索相同、相近、相反的知识,把它们放在一起进行识记与理解。其优点就在于能起到更快地记忆、理解作用。

研究必须充分地占有材料,分析它的各种发展形式,探寻这些形式的内在联系。——马克思



转载自Tetraph:
http://www.tetraph.com/blog/study/study-method/

Saturday 7 November 2015

Daily mail Registration Page Unvalidated Redirects and Forwards Web Security Problem

Daily mail Registration Page Unvalidated Redirects and Forwards Web Security Problem


Website Description:

"The Daily Mail is a British daily middle-market tabloid newspaper owned by the Daily Mail and General Trust. First published in 1896 by Lord Northcliffe, it is the United Kingdom's second biggest-selling daily newspaper after The Sun. Its sister paper The Mail on Sunday was launched in 1982. Scottish and Irish editions of the daily paper were launched in 1947 and 2006 respectively. The Daily Mail was Britain's first daily newspaper aimed at the newly-literate "lower-middle class market resulting from mass education, combining a low retail price with plenty of competitions, prizes and promotional gimmicks", and was the first British paper to sell a million copies a day. It was at the outset a newspaper for women, the first to provide features especially for them, and as of the second-half of 2013 had a 54.77% female readership, the only British newspaper whose female readers constitute more than 50% of its demographic. It had an average daily circulation of 1,708,006 copies in March 2014. Between July and December 2013 it had an average daily readership of approximately 3.951 million, of whom approximately 2.503 million were in the ABC1 demographic and 1.448 million in the C2DE demographic. Its website has more than 100 million unique visitors per month." (Wikipedia)


One of its website's Alexa rank is 93 on January 01 2015. The website is one of the most popular websites in the United Kingdom.







(1) Vulnerability Description:

Daily online websites have a cyber security problem. Hacker can exploit it by Open Redirect (Unvalidated Redirects and Forwards) attacks. During the tests, all Daily mail websites (Daily Mail, Mail on Sunday & Metro media group) use the same mechanism. These websites include dailymail.co.uk, thisismoney.co.uk, and mailonsunday.co.uk.










































Google Dork:

"Part of the Daily Mail, The Mail on Sunday & Metro Media Group"


The vulnerability occurs at "&targetUrl" parameter in "logout.html?" page, i.e.

http://www.dailymail.co.uk/registration/logout.html?targetUrl=http%3A%2F%2Fgoogle.com



(2.1) Use the following tests to illustrate the scenario painted above.


The redirected webpage address is "http://diebiyi.com/articles". Can suppose that this webpage is malicious.


Vulnerable URLs:

http://www.dailymail.co.uk/registration/logout.html?targetUrl=http%3A%2F%2Fdailymail.co.uk
http://www.thisismoney.co.uk/registration/logout.html?targetUrl=http%3A%2F%2Fhao123.com/
http://www.mailonsunday.co.uk/registration/logout.html?targetUrl=http%3A%2F%2Fpinterest.com


POC Code:

http://www.dailymail.co.uk/registration/logout.html?targetUrl=http%3A%2F%2Fdiebiyi.com/articles
http://www.thisismoney.co.uk/registration/logout.html?targetUrl=http%3A%2F%2Fdiebiyi.com/articles
http://www.mailonsunday.co.uk/registration/logout.html?targetUrl=http%3A%2F%2Fdiebiyi.com/articles




POC Video:

https://www.youtube.com/watch?v=AU-HJGe5BWE&feature=youtu.be



Blog Detail:

http://tetraph.com/security/website-test/daily-mail-url-redirection/
http://securityrelated.blogspot.com/2015/10/daily-mail-registration-page.html
https://vulnerabilitypost.wordpress.com/2015/10/30/daily-mail-open-redirect/




(2.2) The program code flaw can be attacked without user login. Tests were performed on Microsoft IE (9 9.0.8112.16421) of Windows 8, Mozilla Firefox (37.0.2) & Google Chromium 42.0.2311 (64-bit) of Ubuntu (14.04.2),and Apple Safari 6.1.6 of Mac OS X v10.9 Mavericks.

These bugs were found by using URFDS (Unvalidated Redirects and Forwards Detection System).







(2) Description of Open Redirect:

Here is the description of Open Redirect: "A web application accepts a user-controlled input that specifies a link to an external site, and uses that link in a Redirect. This simplifies phishing attacks. An http parameter may contain a URL value and could cause the web application to redirect the request to the specified URL. By modifying the URL value to a malicious site, an attacker may successfully launch a phishing scam and steal user credentials. Because the server name in the modified link is identical to the original site, phishing attempts have a more trustworthy appearance." (From CWE)





(3) Vulnerability Disclosure:

These vulnerabilities have not been patched.




Discover and Reporter:

Wang Jing, Division of Mathematical Sciences (MAS), School of Physical and Mathematical Sciences (SPMS), Nanyang Technological University (NTU), Singapore. (@justqdjing)
http://www.tetraph.com/wangjing








Reference:

https://cxsecurity.com/issue/WLB-2015110028
http://computerobsess.blogspot.com/2015/11/daily-mail-open-redirect.html
http://itinfotech.tumblr.com/post/132726134291/ithut-daily-mail-registration-page-unvalidated
http://itsecurity.lofter.com/post/1cfbf9e7_8d45d37
https://inzeed.wordpress.com/2015/11/07/daily-mail-registration-page
http://webtechhut.blogspot.com/2015/11/daily-mail-registration-page.html
https://community.webroot.com/t5/Security-Industry-News/The-Telegraph-and-Daily-Mail

Daily Mail Online Website XSS Cyber Security Zero-Day Vulnerability

Daily Mail Online Website XSS Cyber Security Zero-Day Vulnerability


Website Description:

"The Daily Mail is a British daily middle-market tabloid newspaper owned by the Daily Mail and General Trust. First published in 1896 by Lord Northcliffe, it is the United Kingdom's second biggest-selling daily newspaper after The Sun. Its sister paper The Mail on Sunday was launched in 1982. Scottish and Irish editions of the daily paper were launched in 1947 and 2006 respectively. The Daily Mail was Britain's first daily newspaper aimed at the newly-literate "lower-middle class market resulting from mass education, combining a low retail price with plenty of competitions, prizes and promotional gimmicks", and was the first British paper to sell a million copies a day. It was at the outset a newspaper for women, the first to provide features especially for them, and as of the second-half of 2013 had a 54.77% female readership, the only British newspaper whose female readers constitute more than 50% of its demographic. It had an average daily circulation of 1,708,006 copies in March 2014. Between July and December 2013 it had an average daily readership of approximately 3.951 million, of whom approximately 2.503 million were in the ABC1 demographic and 1.448 million in the C2DE demographic. Its website has more than 100 million unique visitors per month." (Wikipedia)


Domain Name:

http://www.dailymail.co.uk/


The Alexa rank of it is 93 on January 01 2015. It is one of the most popular websites in the United Kingdom.







(1) Vulnerability description:

Daily Mail has a security problem. Criminals can exploit it by XSS attacks.

The vulnerability occurs at "reportAbuseInComment.html?" page with "&commentId" parameter, i.e.

http://www.dailymail.co.uk/home/reportAbuseInComment.html?articleId=346288&commentId=877038


POC Code:

http://www.dailymail.co.uk/home/reportAbuseInComment.html?articleId=346288&commentId="><img src=x onerror=prompt('justqdjing')>























The vulnerability can be attacked without user log in. Tests were performed on Mozilla Firefox (34.0) in Ubuntu (14.04) and Microsoft IE (9.0.15) in Windows 7.




Poc Video:

https://www.youtube.com/watch?v=Oig-ZrlJDf8&feature=youtu.be



Blog Detail:

http://tetraph.com/security/web-security/daily-mail-xss-bug/
http://securityrelated.blogspot.com/2015/10/daily-mail-online-website-xss-cyber.html
https://vulnerabilitypost.wordpress.com/2015/10/30/daily-mail-xss/





(2) What is XSS?

"Cross-site scripting (XSS) is a type of computer security vulnerability typically found in web applications. XSS enables attackers to inject client-side script into web pages viewed by other users. A cross-site scripting vulnerability may be used by attackers to bypass access controls such as the same-origin policy. Cross-site scripting carried out on websites accounted for roughly 84% of all security vulnerabilities documented by Symantec as of 2007. Their effect may range from a petty nuisance to a significant security risk, depending on the sensitivity of the data handled by the vulnerable site and the nature of any security mitigation implemented by the site's owner." (Wikipedia)






(3) Vulnerability Disclosure:

This vulnerability has been patched.






Discoved and Disclosured By:

Wang Jing, Division of Mathematical Sciences (MAS), School of Physical and Mathematical Sciences (SPMS), Nanyang Technological University (NTU), Singapore. (@justqdjing)
http://www.tetraph.com/wangjing







Reference:

https://packetstormsecurity.com/files/134189/Daily-Mail-Unvalidated-Redirect
http://news.softpedia.com/news/the-telegraph-and-daily-mail-fix-xss
https://www.secnews.gr/dailymail_open_redirect_bug
http://whitehatview.tumblr.com/post/132726489926/daily-mail-xss
http://sys-secure.es/daily-mail-registration-page-unvalidated
http://itsecuritynews.info/tag/jing-wang/
http://itsecurity.lofter.com/post/1cfbf9e7_8d45d6b
http://computerobsess.blogspot.com/2015/11/daily-mail-xss.html
https://computertechhut.wordpress.com/2015/11/04/daily-mail-xss/
http://marc.info/?l=full-disclosure&m=144651836427184&w=4


Daily Mail Registration Page Unvalidated Redirects and Forwards & XSS Web Security Problem

Daily Mail Registration Page Unvalidated Redirects and Forwards & XSS Web Security Problem


Website Description:
"The Daily Mail is a British daily middle-market tabloid newspaper owned by the Daily Mail and General Trust. First published in 1896 by Lord Northcliffe, it is the United Kingdom's second biggest-selling daily newspaper after The Sun. Its sister paper The Mail on Sunday was launched in 1982. Scottish and Irish editions of the daily paper were launched in 1947 and 2006 respectively. The Daily Mail was Britain's first daily newspaper aimed at the newly-literate "lower-middle class market resulting from mass education, combining a low retail price with plenty of competitions, prizes and promotional gimmicks", and was the first British paper to sell a million copies a day. It was at the outset a newspaper for women, the first to provide features especially for them, and as of the second-half of 2013 had a 54.77% female readership, the only British newspaper whose female readers constitute more than 50% of its demographic. It had an average daily circulation of 1,708,006 copies in March 2014. Between July and December 2013 it had an average daily readership of approximately 3.951 million, of whom approximately 2.503 million were in the ABC1 demographic and 1.448 million in the C2DE demographic. Its website has more than 100 million unique visitors per month." (Wikipedia)


One of its website's Alexa rank is 93 on January 01 2015. The website is one of the most popular websites in the United Kingdom.


The Unvalidated Redirects and Forwards problem has not been patched, while the XSS problem has been patched.






(1) Daily mail Registration Page Unvalidated Redirects and Forwards Web Security Problem


(1.1) Vulnerability Description:
Daily online websites have a cyber security problem. Hacker can exploit it by Open Redirect (Unvalidated Redirects and Forwards) attacks. During the tests, all Daily mail websites (Daily Mail, Mail on Sunday & Metro media group) use the same mechanism. These websites include dailymail.co.uk, thisismoney.co.uk, and mailonsunday.co.uk.















































Google Dork:
"Part of the Daily Mail, The Mail on Sunday & Metro Media Group"


The vulnerability occurs at "&targetUrl" parameter in "logout.html?" page, i.e.
http://www.dailymail.co.uk/registration/logout.html?targetUrl=http%3A%2F%2Fgoogle.com






(1.2.1) Use the following tests to illustrate the scenario painted above.

The redirected webpage address is "http://diebiyi.com/articles". Can suppose that this webpage is malicious.

Vulnerable URLs:


POC Code:




POC Video:



Blog Details:



(1.2.2) The program code flaw can be attacked without user login. Tests were performed on Microsoft IE (9 9.0.8112.16421) of Windows 8, Mozilla Firefox (37.0.2) & Google Chromium 42.0.2311 (64-bit) of Ubuntu (14.04.2),and Apple Safari 6.1.6 of Mac OS X v10.9 Mavericks.

These bugs were found by using URFDS (Unvalidated Redirects and Forwards Detection System).





(1.2) Description of Open Redirect:
Here is the description of Open Redirect: "A web application accepts a user-controlled input that specifies a link to an external site, and uses that link in a Redirect. This simplifies phishing attacks. An http parameter may contain a URL value and could cause the web application to redirect the request to the specified URL. By modifying the URL value to a malicious site, an attacker may successfully launch a phishing scam and steal user credentials. Because the server name in the modified link is identical to the original site, phishing attempts have a more trustworthy appearance." (From CWE)




(1.3) Vulnerability Disclosure:
These vulnerabilities have not been patched.






(2) Daily Mail Website XSS Cyber Security Zero-Day Vulnerability


(2.1) Vulnerability description:
DailyMail has a security problem. Criminals can exploit it by XSS attacks.

The vulnerability occurs at "reportAbuseInComment.html?" page with "&commentId" parameter, i.e.
http://www.dailymail.co.uk/home/reportAbuseInComment.html?articleId=346288&commentId=877038



POC Code:
http://www.dailymail.co.uk/home/reportAbuseInComment.html?articleId=346288&commentId="><img src=x onerror=prompt('justqdjing')>




The vulnerability can be attacked without user login. Tests were performed on Mozilla Firefox (34.0) in Ubuntu (14.04) and Microsoft IE (9.0.15) in Windows 7.
























(2.2) What is XSS?
"Cross-site scripting (XSS) is a type of computer security vulnerability typically found in web applications. XSS enables attackers to inject client-side script into web pages viewed by other users. A cross-site scripting vulnerability may be used by attackers to bypass access controls such as the same-origin policy. Cross-site scripting carried out on websites accounted for roughly 84% of all security vulnerabilities documented by Symantec as of 2007. Their effect may range from a petty nuisance to a significant security risk, depending on the sensitivity of the data handled by the vulnerable site and the nature of any security mitigation implemented by the site's owner." (Wikipedia)




(2.3) Vulnerability Disclosure:
This vulnerability has been patched.




Discover and Reporter:
Wang Jing, Division of Mathematical Sciences (MAS), School of Physical and Mathematical Sciences (SPMS), Nanyang Technological University (NTU), Singapore. (@justqdjing)