View source for BAA 18S

==Introduction==

For Fall 2017 and Spring 2018, candidates using BattleActs had a '''significantly higher''' pass rate than the overall candidate pool. For Fall 2018 however, almost all Exam 6-Canada candidates are using BattleActs. That means that going forward, there will be little difference between the BattleActs pass rate and the overall pass rate.

The BattleActs system tracks basic information about users. We do this to create a '''profile''' of a successful candidate. You can then adjust your study strategy to align more closely with this optimal profile.

Our original hypothesis was that passing would be strongly correlated with BRQ ''(Battle-Readiness Quotient)'', but the correlation has not been as strong as expected. This is discussed below. ''(Things in the real world never work out as well as in a textbook!)''

==Executive Summary==

Here is the current ranking of variables with respect to probability of passing:

:{| class="wikitable"
|-
! category !! variable !! importance
|-
|| forum            || # of posts to the BattleActs forum             || high
|-
|| logins           || # of days with at least 1 login to BattleActs  || high
|-
|| logins           || total # of logins to the system                || moderate
|-
| BRQ               || BRQ on the day before the exam                 || moderate
|-
| registration date || date of BattleActs registration                || low
|}

Successful candidates have a higher level of '''engagement'''. We used [# of posts to forum] as a proxy for '''strength''' of engagement, and [# of days with at least 1 login] as a proxy for '''frequency''' of engagement.

* Of the approximately 40 candidates who used BattleActs for Spring 2018, 10 posted to the forum at least once. Of those 10, 80% passed the exam ''(although many of our passing candidates did not post at all.)''
* Of the candidates who signed in to BattleActs on 86 or more different days, 70% passed. ''(roughly 3 months of daily study)''
* Of the candidates who signed in to BattleActs on 63 or more different days, 60% passed. ''(roughly 2 months of daily study)''

So, keeping in mind that our data is thin after only 2 sittings, a profile of a successful candidate is one who <u>logs in</u> '''consistently''' and <u>posts</u> '''at least occasionally''' to the forum.

==Data Quality==

Obviously the quality of these results depends on the quality of the data. At this early stage in the development of the BattleActs system, our data has the following shortcomings:

# Our data is still relatively '''thin'''. ''(We have data only for the beta-test for Fall 2017, and for the live version for Spring 2018.)''
# We generally don't know whether a candidate had '''prior attempts''' at Exam 6. This would be a confounding variable.
# There is an element of '''luck''' in passing this exam. Many candidates have scores that are very, very close to the pass mark, and just a tiny bit of luck could have lifted them from a fail to a pass. This element of luck can skew our data.

As we develop the system and gain more users, the quality of data will improve.

==Detailed Comments==

===BRQ===

The most significant point that deserves elaboration is the relative importance of the BRQ ''(Battle-Readiness Quotient)''. The '''intent''' of the BRQ is to provide a continuous assessment of your readiness to take the exam. We were hoping  to find a strong correlation between BRQ and probability of passing, but the correlation is only moderate. Here are a few possible explanations:

* Most of the BattleCards are '''self-scored''', so you have to be <u>brutally honest</u>. It's very tempting to always click "correct" because there is instantaneous positive feedback! You see the <span style="color: red;">'''+27'''</span> or whatever in the navigation bar and your BRQ goes up by that amount. Now, you guys are all smart and I don't think you're being intentionally dishonest, but there is skill involved in using the BattleActs system effectively. More on this later in the article.
* Some candidates have told us they '''don't like flash cards'''. They do virtually all of their studying using the wiki articles and the examiner's reports. We had one successful candidate who had a BRQ of only 15%. ''(Note that we are continually revising and improving the BattleCards, and we're noticing that people are generally using them more than during the original beta-test.)''

In any event, we hope that through better support in using BattleActs effectively, the correlation between BRQ and passing will become stronger. ''(But it will never be perfect because there will always be external influences that we can neither measure nor control.)''

===Forum & Logins===

As stated above, the most significant predictors in our data are forum posts and consistency of logins.

* If you're really engaged in the material, you'll likely have questions. And if you have questions, you can get them answered on the forum. That's the connection. Even if you don't post to the forum, you can glean information from other people's posts. If you aren't at least registered for the forum, you're missing out on a key element in preparing for the exam.
* The correlation between passing and [# of days with at least 1 login] is pretty obvious. Interestingly, it is a better predictor than [total # of logins]. In a very rough sense, a high value for [total # of logins] may indicate a tendency to cram ''(not good!)'' whereas [# of days with at least 1 login] may indicate consistency. In other words, a certain number of study hours spread over a longer period is better than the same number of hours over a shorter period.

The positive effect of spreading your study hours over a greater number is days is more pronounced when you're learning a lot of material. If you only had to learn 10 facts, you could cram that into your brain the night before. But for Exam 6, you have to learn something like 1500 facts. That requires a disciplined, consistent, and long-term approach.

===Registration Date===

Candidates who registered in January for the Spring 2018 exam seemed to have a small advantage in terms of passing. At the same time, there were a handful of passing candidates with March registration dates. It seems plausible that these late-registering passing candidates may have had prior attempts because there aren't many people who could start from scratch and pass with only 6 weeks of studying. Anyway, a good rule of thumb is:

* Register by the end of '''January''' for the Spring exam.
* Register by the end of '''July''' for the Fall exam.

This gives you 3 months to study, and if you study <u>effectively</u>, you can give yourself a good chance to pass. ''(Of course, giving yourself 4 or 5 months provides a margin for error, just in case you get sick or get busy at work.)''

==Common Reasons for Not Passing==

For returning BattleActs users, we provide a free analysis of grade reports. From those grade reports, we've identified common reasons for not passing. ''(It may be helpful to keep the [[Old Exams|Spring 2018 BattleTable]] open in a separate tab for easy reference.)'' '''In order of importance''', the reasons for not passing relate to the following areas:

# Memorization of facts
# Time management during the exam
# Bloom’s Taxonomy
# Calculations

We'll deal with each item in turn and offer suggestions on how to improve.

===Memorization===

This is the <span style="color: red;>'''number one reason''' </span> people fail Exam 6.

* If you have to memorize 10 facts, you can probably do it in 10 minutes. But if you have to memorize 20 facts, it will take '''more''' than twice as long, maybe 25 minutes. Now extrapolate that to Exam 6 where you have to memorize something like 1500 facts. To learn these facts <u>reliably</u>, you need to go over each one dozens of times, and you have to do it carefully. It's very easy to "review" a BattleCard too quickly and fool yourself into thinking you know the answer when you really don't.
* '''This cannot be stressed enough!''' Don't fall into the trap of doing the BattleCards <u>too quickly</u>. There's a difference between merely <u>recognizing</u> a question and truly <u>thinking</u> through the answer. A good habit is to write the answer on a piece of paper. That takes way longer but it's a truer test of whether you know it versus just saying to yourself, "Yup, I've done this a million times. I totally know it!" After writing out 10 or 20 BattleCard answers in full, you can switch to saying the answer out loud, or train yourself to recite it mentally. But even if you do it mentally, each BattleCard could take anywhere from 10 seconds to 30 or 40 seconds to do properly.

BattleActs was designed primarily to help you memorize the facts. If you've used BattleActs, you already know that the papers are ranked by how often they've appeared on prior exams. This is the basis of the [[Suggested Study Schedule]]. And your BRQ is designed to indicate how well you're progressing through the material. '''Let the BRQ do its job!'''

* If in doubt, mark a BattleCard '''''wrong'''''. Then use ''Level 3: Custom Battles'' to select only those '''''wrong''''' BattleCards from a particular paper. ''(Do this using <u>Optional Setting B</u> and selecting "<span style="color: red;>'''red'''</span>" for correctness status.)''
* Then go through those BattleCards carefully, as described above, but '''don't''' mark them "correct" unless you've gone through them several times over the course of a few days.

It takes much longer than you think to memorize the facts <u>reliably</u>.

Another trick is to use the '''Shuffle''' button on the BattleCard page. Once you've done a set of BattleCards several times, you'll solidify your knowledge if you do them in a <u>different order</u> every so often. An established principle of learning is that you have to practice in different ways. If you play a musical instrument, you might know that practicing a piece at different tempos and/or using different rhythms will enhance your playing proficiency, versus playing it at the same tempo in the same way every time.

===Time Management===

Running out of time is a heart-breaking way to fail an exam. If you're well-prepared, then time management will be a bigger issue because you'll know how to answer most ''(or all)'' of the questions. You have to make sure you finish, even if that means writing less than you want to for a particular question. If you <u>have</u> the time, it's better to write more than less, but <u>not</u> at the expense of missing a question you know how to do.

There is a detailed discussion of time management in the wiki article [[On BattleDay]]. If you don't like my method, invent your own. But you have to take time management seriously!  '''Don't wing it on exam day!'''

===Bloom's Taxonomy===

Examples of Bloom's Taxonomy questions from Spring 2018 are #3, #9, #11, #24. ''(See [[Old Exams|Spring 2018 BattleTable]].)'' This is something that's hard to teach because being good at answering these types of questions improves mainly through experience. Very briefly, this is what these questions are about:

: '''#3''': analyze a scenario related to credit scoring
: '''#9''': create a flood insurance program
: '''#11''': analyze a scenario for potential insolvency
: '''#24''': analyze a given DCAT scenario and evaluate the actions of the actuary

These questions are '''based''' on facts, but facts alone aren't enough to give a complete answer. The best advice here is to study the answers in the examiner's reports for these types of questions, and just try to <u>absorb</u> how these problems are done.

One concrete piece of advice, however, is to first try to recognize which facts may be relevant to the answer. For example in question #3 on credit scoring, you might recall that <u>privacy</u> and <u>customer consent</u> are important concepts. '''Write that down!''' Then think about how those concepts might relate to the specific question that was asked. Be imaginative. There are many variations of correct answers for these kinds of questions.

'''Side note''': There's a great math book by the famous Hungarian mathematician ''George Polya'' called ''<u>How to Solve It</u>''. It's all about heuristics for solving problems. He doesn't use the term Bloom's Taxonomy, but that's essentially what it's about. You have to put ideas together in an imaginative way. Look it up sometime.

===Calculations===

This is the last item on the list and it is '''already an area of strength''' for actuaries. The only piece of advice I have is to practice more. BattleActs has practice templates for many of the common types of calculation problems on the exam. Once you're good at the practice templates, make sure you can do all the recent exam problems. But it's not enough to do a problem once or twice. I've found that to really get it into my head, I have to do each problem many times over the course of my study. Going back to the music analogy, you know that to learn a piece of music you have to practice it daily ''(and carefully)'' over a period of weeks ''(or months!)''

==Closing Thoughts==

The single best piece of advice I can give you is to find a way to '''stay engaged'''. That will support '''consistent''' and '''effective''' studying. Try not to study when you're tired because that's usually a waste of time. Take nap instead. And don't be too hard on yourself. If you start early and follow a plan, you'll be in good shape. You've made it this far in the exam process. You '''will''' get through it!

'''''Slay the beast.''''' :-)