Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factors

Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factors Zeyu Yun ∗2 Yubei Chen ∗1,2 Bruno A Olshausen 2,4 Yann LeCun 1,3 1 Facebook AI Research 2 Berkeley AI Research (BAIR), UC Berkeley 3 New York University 4 Redwood Center for Theoretical Neuroscience, UC Berkeley Abstract Transformer networks have revolutionized NLP representation learning since they were introduced. Though a great effort has been made to explain the representation in trans- formers, it is widely recognized that our un- derstanding is not sufficient. One important reason is that there lack enough visualiza- tion tools for detailed analysis. In this pa- per, we propose to use dictionary learning to open up these ‘black boxes’ as linear su- perpositions of transformer factors. Through visualization, we demonstrate the hierarchi- cal semantic structures captured by the trans- former factors, e.g., word-level polysemy dis- ambiguation, sentence-level pattern formation, and long-range dependency.While some of these patterns confirm the conventional prior linguistic knowledge, the rest are rela- tively unexpected, which may provide new in- sights. We hope this visualization tool can bring further knowledge and a better under- standing of how transformer networks work. The code is available athttps://github. com/zeyuyun1/TransformerVis. 1 Introduction Though the transformer networks (Vaswani et al., 2017; Devlin et al., 2018) have achieved great suc- cess, our understanding of how they work is still fairly limited. This has triggered increasing efforts to visualize and analyze these “black boxes”. Be- sides a direct visualization of the attention weights, most of the current efforts to interpret transformer models involve “probing tasks”. They are achieved by attaching a light-weighted auxiliary classifier at the output of the target transformer layer. Then only the auxiliary classifier is trained for well- known NLP tasks like part-of-speech (POS) Tag- ging, Named-entity recognition (NER) Tagging, ∗ equal contribution.Correspondence to:Zeyu Yun<chobitstian@berkeley.edu>,YubeiChen <yubeic@fb.com, berkeley.edu> Syntactic Dependency, etc. Tenney et al. (2019) and Liu et al. (2019) show transformer models have excellent performance in those probing tasks. These results indicate that transformer models have learned the language representation related to the probing tasks. Though the probing tasks are great tools for interpreting language models, their lim- itation is explained in Rogers et al. (2020). We summarize the limitation into three major points: •Most probing tasks, like POS and NER tag- ging, are too simple. A model that performs well in those probing tasks does not reflect the model’s true capacity. •Probing tasks can only verify whether a cer- tain prior structure is learned in a language model. They can not reveal the structures be- yond our prior knowledge. • It’s hard to locate where exactly the related linguistic representation is learned in the trans- former. Efforts are made to remove those limitations and make probing tasks more diverse. For instance, Hewitt and Manning (2019) proposes “structural probe”, which is a much more intricate probing task. Jiang et al. (2020) proposes to generate spe- cific probing tasks automatically. Non-probing methods are also explored to relieve the last two limitations. For example, Reif et al. (2019) visu- alizes embedding from BERT using UMAP and shows that the embeddings of the same word un- der different contexts are separated into different clusters. Ethayarajh (2019) analyzes the similarity between embeddings of the same word in different contexts. Both of these works show transformers provide a context-specific representation. Faruqui et al. (2015); Arora et al. (2018); Zhang et al. (2019) demonstrate how to use dictionary learning to explain, improve, and visualize the un- contextualized word embedding representations. In arXiv:2103.15949v2 [cs.CL] 4 Apr 2023 this work, we propose to use dictionary learning to alleviate the limitations of the other transformer interpretation techniques. Our results show that dic- tionary learning provides a powerful visualization tool, leading to some surprising new knowledge. 2 Method Hypothesis: contextualized word embedding as a sparse linear superposition of transformer factors. It is shown that word embedding vectors can be factorized into a sparse linear combination of word factors (Arora et al., 2018; Zhang et al., 2019), which correspond to elementary semantic meanings. An example is: apple=0.09“dessert” + 0.11“organism” + 0.16 “fruit” + 0.22“mobile&IT” + 0.42“other”. We view the latent representation of words in a transformer as contextualized word embedding. Similarly, we hypothesize that a contextualized word embedding vector can also be factorized as a sparse linear superposition of a set of elemen- tary elements, which we calltransformer factors. The exact definition will be presented later in this section. Figure 1: Building block (layer) of transformer Due to the skip connections in each of the trans- former blocks, we hypothesize that the representa- tion in any layer would be a superposition of the hi- erarchical representations in all of the lower layers. As a result, the output of a particular transformer block would be the sum of all of the modifications along the way. Indeed, we verify this intuition with the experiments. Based on the above observa- tion, we propose to learn a single dictionary for the contextualized word vectors from different layers’ output. To learn a dictionary of transformer factors with non-negative sparse coding. Given a set of tokenized text sequences, we col- lect the contextualized embedding of every word using a transformer model. We define the set of all word embedding vectors fromlth layer of trans- former model asX (l) . Furthermore, we collect the embeddings across all layers into a single set X=X (1) ∪X (2) ∪·∪X (L) . By our hypothesis, we assume each embedding vectorx∈Xis a sparse linear superposition of transformer factors: x= Φα+, s.t. α0,(1) whereΦ∈IR d×m is a dictionary matrix with columnsΦ :,c ,α∈IR m is a sparse vector of coef- ficients to be inferred andis a vector containing independent Gaussian noise samples, which are as- sumed to be small relative tox. Typicallym > d so that the representation isovercomplete. This inverse problem can be efficiently solved by FISTA algorithm (Beck and Teboulle, 2009). The dictio- nary matrixΦcan be learned in an iterative fashion by using non-negative sparse coding, which we leave to the appendix section C. Each columnΦ :,c ofΦis atransformer factorand its corresponding sparse coefficientα c is its activation level. Visualization by top activation and LIME inter- pretation.An important empirical method to visu- alize a feature in deep learning is to use the input samples, which trigger the top activation of the fea- ture (Zeiler and Fergus, 2014). We adopt this con- vention. As a starting point, we try to visualize each of the dimensions of a particular layer,X (l) . Un- fortunately, the hidden dimensions of transformers are not semantically meaningful, which is similar to the uncontextualized word embeddings (Zhang et al., 2019). Instead, we can try to visualize the transformer factors. For a transformer factorΦ :,c and for a layer-l, we denote the 1000 contextualized word vectors with the largest sparse coefficientsα (l) c as X (l) c ⊂X (l) , which correspond to 1000 differ- ent sequences. For example, Figure 3 shows the top 5 words that activated transformer factor-17 Φ :,17 at layer-0, layer-2, and layer-6respectively. Since a contextualized word vector is generally af- fected by many tokens in the sequence, we can use LIME (Ribeiro et al., 2016) to assign a weight to each token in the sequence to identify their relative importance toα c . The detailed method is left to Section 3. To determine low-, mid-, and high-level trans- former factors with importance score.As we build a single dictionary for all of the transformer layers, the semantic meaning of the transformer fac- tors has different levels. While some of the factors appear in lower layers and continue to be used in the later stages, the rest of the factors may only be activated in the higher layers of the transformer net- work. A central question in representation learning is: “where does the network learn certain informa- tion?” To answer this question, we can compute an “importance score” for each transformer factor Φ :,c at layer-lasI (l) c .I (l) c is the average of the largest 1000 sparse coefficientsα (l) c ’s, which cor- respond toX (l) c . We plot the importance scores for each transformer factor as a curve is shown in Figure 2. We then use these importance score (IS) curves to identify which layer a transformer factor emerges. Figure 2a shows an IS curve peak in the earlier layer. The corresponding transformer factor emerges in the earlier stage, which may capture lower-level semantic meanings. In contrast, Fig- ure 2b shows a peak in the higher layers, which indicates the transformer factor emerges much later and may correspond to mid- or high-level seman- tic structures. More subtleties are involved when distinguishing between mid-level and high-level factors, which will be discussed later. An important characteristic is that the IS curve for each transformer factor is relatively smooth. This indicates if a vital feature is learned in the beginning layers, it won’t disappear in later stages. Instead, it will be carried all the way to the end with gradually decayed weight since many more features would join along the way. Similarly, ab- stract information learned in higher layers is slowly developed from the early layers. Figure 3 and 5 confirm this idea, which will be explained in the next section. 3 Experiments and Discoveries We use a 12-layer pre-trained BERT model (Pre; Devlin et al., 2018) and freeze the weights. Since we learn a single dictionary of transformer factors for all of the layers in the transformer, we show that these transformer factors correspond to different levels of semantic or syntactic patterns. The pat- terns can be roughly divided into three categories: (a)(b) Figure 2: Importance score (IS) across all layers for two different transformer factors. (a) This figure shows a typical IS curve of a transformer factor correspond- ing to low-level information. (b) This figure shows a typical IS curve of a transformer factor corresponds to mid-level information. word-level disambiguation, sentence-level pattern formation, and long-range dependency. In the fol- lowing, we provide detailed visualization for each pattern category. Due to the space limit, only a small amount of the factors are demonstrated in the paper. To alleviate the “cherry-picking” bias, we also build a website for the interested readers to play with these results. Low-level: word-level polysemy disambigua- tion.While the input embedding of a token con- tains polysemy, we find transformer factors with early IS curve peaks usually correspond to a spe- cific word-level meaning. By visualizing the top activation sequences, we can see how word-level disambiguation is gradually developed in a trans- former. We show how the disambiguation effect devel- ops progressively through each layer in Figure 3. In Figure 3, the top 5 activated words and their contexts for transformer factorΦ :,30 in different layers are listed. The top activated words in layer 0 contain the word “left” varying senses, which is being mostly disambiguated in layer 2 albeit not completely. In layer 4, the word “left” is fully disambiguated since the top-activated word con- tains only “left” with the word sense “leaving, exit- ing.” We also show more examples of those types of transformer factors in Table 1: for each trans- former factor, we list out the top 3 activated words and their contexts in layer 4. As shown in the table, nearly all top-activated words are disambiguated into a single sense. Further, we can quantify the quality of the disam- biguation ability of the transformer model. In the example above, since the top 1000 activated words (a) layer 0(b) layer 2(c) layer 6 Figure 3: Visualization of a low-level transformer factor,Φ :,30 at different layers. (a), (b) and (c) are the top- activated words and contexts forΦ :,30 in layer-0,2and4respectively. We can see that at layer-0, this transformer factor corresponds to word vectors that encode the word “left” with different senses. In layer-2, a majority of the top activated words “left” correspond to a single sense, "leaving, exiting." In layer 4, all of the top-activated words “left” have corresponded to the same sense, "leaving, exiting." Due to space limitations, we invite the readers to use our website to see more of those disambiguation effects. Top 3 activated words and their contextsExplanation Φ :,2 •that snare shot sounded like somebody’ d kicked open the door to your mind". •i became very frustrated with that and finally made up my mind to start getting back into things." • when evita asked for more time so she could make up her mind, the crowd demanded," ¡ ahora, evita,< • Word “mind” • Noun •Definition: the element of a person that enables them to be aware of the world and their ex- periences. Φ :,16 •nington joined the five members xero and the band was renamed to linkin park. •times about his feelings about gordon, and the price family even sat away from park’ s supporters during the trial itself. • on 25 january 2010, the morning of park’ s 66th birthday, he was found hanged and unconscious in his • Word “park” • Noun •Definition: a common first and last name Φ :,30 •saying that he has left the outsiders, kovu asks simba to let him join his pride •eventually, all boycott’ s employees left, forcing him to run the estate without help. •the story concerned the attempts of a scientist to photograph the soul as it left the body. • Word “left" • Verb • Definition: leaving, exiting Φ :,33 •forced to visit the sarajevo television station at night and to film with as little light as possible to avoid the attention of snipers and bombers. • by the modest, cream@-@ colored attire in the airy, light@-@ filled clip. • the man asked her to help him carry the case to his car, a light@-@ brown volkswagen beetle. • Word “light” • Noun • Definition: the natural agent that stimulates sight and makes things visible Table 1: Several examples of low-level transformer factors. Their top-activated words in layer 4 are marked blue, and the corresponding contexts are shown as examples for each transformer factor. As shown in the table, nearly all of the top-activated words are disambiguated into a single sense. Please note the last example ofΦ :,33 is a rare exception, the reader may check the appendix to see a more complete list. More examples, top-activated words and contexts are provided in Appendix. and contexts are “left” with only the word sense “leave, exiting”, we can assume “left” when used as a verb, triggers higher activation inΦ :,30 than “left” used as other sense of speech. We can verify this hypothesis using a human-annotated corpus: Brown corpus (Francis and Kucera, 1979). In this corpus, each word is annotated with its correspond- ing part-of-speech. We collect all the sentences contains the word “left” annotated as a verb in one set and sentences contains “left” annotated as other part-of-speech. As shown in Figure 4a, in layer 0, the average activation ofΦ :,30 for the word “left” marked as a verb is no different from “left” as other senses. However, at layer 2, “left” marked as a verb triggers a higher activation ofΦ :,30 . In layer 4, this difference further increases, indicating dis- ambiguation develops progressively across layers. In fact, we plot the activation of “left” marked as verb and the activation of other “left” in Figure 4b. In layer 4, they are nearly linearly separable by this (a)(b) Figure 4: (a) Average activation ofΦ :,30 for word vector “left” across different layers. (b) Instead of averaging, we plot the activation of all “left” with different contexts in layer-0,2, and4. Random noise is added to the y-axis to prevent overplotting. The activation ofΦ :,30 for two different word senses of “left” is blended together in layer-0. They disentangle to a great extent in layer-2and nearly separable in layer-4by this single dimension. (a) layer 4(b) layer 6(c) layer 8 Figure 5: Visualization of a mid-level transformer factor. (a), (b), (c) are the top 5 activated words and contexts for this transformer factor in layer-4,6, and8respectively. Again, the position of the word vector is marked blue. Please notice that sometimes only a part of a word is marked blue. This is due to that BERT uses word-piece tokenizer instead of whole word tokenizer. This transformer factor corresponds to the pattern of “consecutive adjective”. As shown in the figure, this feature starts to develop at layer-4and fully develops at layer-8. Precision (%) Recall (%) F1 score (%) Average perceptron POS tagger 92.795.594.1 Finetuned BERT base model for POS task 97.595.296.3 Logistic regression clas- sifier with activation of Φ :,30 at layer 4 97.295.896.5 Table 2: Evaluation of binary POS tagging task: predict whether or not “left” in a given context is a verb. single feature. Since each word “left” corresponds to an activation value, we can perform a logistic regression classification to differentiate those two types of “left”. From the result shown in Figure 4a, it is pretty fascinating to see that the disambigua- tion ability of justΦ :,30 is better than the other two classifiers trained with supervised data. This result confirms that disambiguation is indeed done in the early part of pre-trained transformer model and we are able to detect it via dictionary learning. Mid level: sentence-level pattern formation.We find most of the transformer factors, with an IS curve peak after layer 6, capture mid-level or high- level semantic meanings. In particular, the mid- level ones correspond to semantic patterns like phrases and sentences pattern. We first show two detailed examples of mid-level transformer factors. Figure 5 shows a transformer factor that detects the pattern of consecutive usage of adjectives. This pattern starts to emerge at layer 4, develops at layer 6, and becomes quite reliable at layer 8. Figure 6 shows a transformer factor, which corresponds to a pretty unexpected pattern: “unit exchange”, e.g., 56 inches (140 cm). Although this exact pattern only starts to appear at layer 8, the sub-structures that make this pattern, e.g., paren- thesis and numbers, appear to trigger this factor in layers 4 and 6. Thus this transformer factor is also (a) layer 4(b) layer 6(c) layer 8 Figure 6: Another example of a mid-level transformer factor visualized at layer-4,6, and8. The pattern that cor- responds to this transformer factor is “unit exchange”. Such a pattern is somewhat unexpected based on linguistic prior knowledge. 2 example words and their contexts with high activationPatterns L4 (%) L6 (%) L8 (%) L10 (%) Φ :,13 • the steel pipeline was about 20 ° f(- 7 ° c) degrees. •hand( 56 to 64 inches( 140 to 160 cm)) war horse is that it was a Unit exchange with paren- theses 0064.595.5 Φ :,42 • he died at the hospice of lancaster county from heart •holly’ s drummer carl bunch suffered frostbite to his toes( while aboard the ailments on 23 june 2007. Something unfortunate happened 94.0100100100 Φ :,50 •hurricane pack 1 was a revamped version of story mode; •in 1998, the categories were retitled best short form music video, and best Doing something again, or making something new again 74.5100100100 Φ :,86 •he finished the 2005 – 06 season with 21 appearances and seven goals. •of an offensive game, finishing off the 2001 – 02 season with 58 points in the 47 games Consecutive years, used in foodball season nam- ing 010085.095.5 Φ :,102 •the most prominent of which was bishop abel muzorewa’ s united african national council •ralambo’ s father, andriamanelo, had established rules of succession by African names99.0100100100 Φ :,125 •music writer jeff weiss of pitchfork describes the" endur- ing image" • club reviewer erik adams wrote that the episode was a perfect mix Describing someone in a paraphrasing style. Name, Career 15.599.010098.5 Φ :,184 •the world wide fund for nature( wwf) announced in 2010 that a biodiversity study from •fm) was halted by the federal communications commis- sion( fcc) due to a complaint that the company buying Institution with abbrevia- tion 015.539.063.0 Φ :,193 •74, 22@,@ 500 vietnamese during 1979 – 92, over 2@,@ 500 bosnian • , the russo@-@ turkish war of 1877 – 88 and the first balkan war in 1913. Time span in years97.095.596.595.5 Φ :,195 •s, hares, badgers, foxes, weasels, ground squirrels, mice, hamsters •-@ watching, boxing, chess, cycling, drama, languages, geography, jazz and other music Consecutiveofnoun (Enumerating) 8.098.5100100 Φ :,225 •technologist at the united states marine hospital in key west, florida who developed a morbid obsession for • 00°,11”, w, near smith valley, nevada. Places in US, follow- ings the convention “city, state" 51.591.591.077.5 Table 3: A list of typical mid-level transformer factors. The top-activation words and their context sequences for each transformer factor at layer-8are shown in the second column. We summarize the patterns of each transformer factor in the third column. The last 4 columns are the percentage of the top 200 activated words and sequences that contain the summarized patterns in layer-4,6,8, and10respectively. gradually developed through several layers. While some mid-level transformer factors verify common semantic or syntactic patterns, there are also many surprising mid-level transformer factors. We list a few in Table 3 with quantitative analysis. For each listed transformer factor, we analyze the top 200 activating words and their contexts in each layer. We record the percentage of those words and contexts that correspond to the factors’ semantic pattern in Table 3. From the table, we see that large Adversarial TextExplainationα 35 (o) album as "full of exhilarating, ecstatic, thrilling, fun and sometimes downright silly songs" The original top-activated word and its context sentence for transformer factor Φ :,35 (not an adversarial text) 9.5 (a) album as "full of delightful, lively, exciting, interesting and sometimes downright silly songs" Replace the adjectives in sentence (o) with different adjectives. 9.2 (b) album as "full of unfortunate, heartbroken, annoying, bor- ing and sometimes downright silly songs" Replace the adjectives in sentence (o) with negative adjectives. 8.2 (c) album as "full of [UNK], [UNK], thrilling, [UNK] and sometimes downright silly songs" Mask the adjectives in sentence (o) with unknown tokens. 5.3 (d) album as "full of thrilling and sometimes downright silly songs" Remove the first three adjectives in sen- tence (o). 7.8 (e) album as "full of natural, smooth, rock, electronic and sometimes downright silly songs" Replace the adjectives in sentence (o) with neutral adjectives. 6.2 (f) each participant starts the battle with one balloon. these can be re@-@ inflated up to four Use a random sentence.0.0 (g) The book is described as "innovative, beautiful and bril- liant". It receive the highest opinion from James Wood We create this sentence that contain the pattern of consecutive adjective. 7.9 Table 4: We construct adversarial texts similar but different to the pattern “Consecutive adjective”. The last column shows the activation ofΦ :,35 , orα (8) 35 , w.r.t. the blue-marked word in layer 8. percentages of top-activated words and contexts do corresponds to the pattern we describe. It also shows most of these mid-level patterns start to de- velop at layer 4 or 6. More detailed examples are provided in the appendix section F. Though it’s still mysterious why the transformer network develops representations for these surprising patterns, we believe such a direct visualization can provide ad- ditional insights, which complements the “probing tasks”. To further confirm a transformer factor does correspond to a specific pattern, we can use con- structed example words and context to probe their activation. In Table 4, we construct several text sequences that are similar to the patterns corre- sponding to a particular transformer factor but with subtle differences. The result confirms that the con- text that strictly follows the pattern represented by that transformer factor triggers a high activation. On the other hand, the closer the adversarial exam- ple to this pattern, the higher activation it receives at this transformer factor. High-level: long-range dependency.High-level transformer factors correspond to those linguistic patterns that span an extended range in the text. Since the IS curves of mid-level and high-level transformer factors are similar, it is difficult to dis- tinguish those transformer factors based on their IS cures. Thus, we have to manually examine the top-activation words and contexts for each trans- former factor to differentiate between mid-level and high-level transformer factors. To ease the process, we choose to use the black-box interpreta- tion algorithmLIME(Ribeiro et al., 2016) to iden- tify the contribution of each token in a sequence. There also exist interpretation tools that specifically leverage the transformer architecture (Chefer et al., 2021, 2020). In the future, one could adapt those interpretation tools, which may potentially provide better visualization. Given a sequences∈S, we can treatα (l) c,i , the activation ofΦ :,c in layer-lat locationi, as a scalar function ofs,f (l) c,i (s). Assume a sequencestrig- gers a high activationα (l) c,i , i.e.f (l) c,i (s)is large. We want to know how much each token (or equivalently each position) inscontributes tof (l) c,i (s). To do so, we generated a sequence setS(s), where each s ′ ∈ S(s)is the same assexcept for that several random positions ins ′ are masked by [‘UNK’] (the unknown token). Then we learns a linear model g w (s ′ )with weightsw∈R T to approximatef(s ′ ), whereTis the length of sentences. This can be solved as a ridge regression: min w∈R T L(f,w,S(s)) +σ‖w‖ 2 2 . The learned weightswcan serve as a saliency map that reflects the “contribution” of each token in the sequences. Like in Figure 7, the color re- flects the weightswat each position. Red means the given position has positive weight and green means negative weight. The magnitude of weight is represented by the intensity. The redder a token is, the more it contributions to the activation of the transformer factor. We leave more implementa- tion and mathematical formulation details of LIME algorithm in the appendix. We provide detailed visualization for two differ- ent transformer factors that show long-range depen- dency in Figure 7, 8. Since visualization of high- level information requires more extended context, we only offer the top two activated words and their contexts for each such transformer factor. Many more will be provided in the appendix section G. We name the pattern for transformer factorΦ :,297 in Figure 7 as “repetitive pattern detector”. All top activated contexts forΦ :,297 contain an obvious repetitive structure. Specifically, the text snippet “can’t get you out of my head" appears twice in the first example, and the text snippet “x class pas- senger, star alliance” appears three times in the sec- ond example. Compared to the patterns we found in the mid-level [6], the high-level patterns like “repetitive pattern detector” are much more abstract. In some sense, the transformer detects if there are two (or multiple) almost identical embedding vec- tors at layer-10without caring what they are. Such behavior might be highly related to the concept proposed in the capsule networks (Sabour et al., 2017; Hinton, 2021). To further understand this be- havior and study how the self-attention mechanism helps model the relationships between the features outlines an interesting future research direction. Figure 8 shown another high-level factor, which detects text snippets related to “the beginning of a biography”. The necessary components, day of birth as month and four-digit years, first name and last name, familial relation, and career, are all mid- level information. In Figure 8, we see that all the information relates to biography has a high weight in the saliency map. Thus, they are all together combined to detect the high-level pattern. Figure 7: Two examples of the high activated words and their contexts for transformer factorΦ :,297 . We also provide the saliency map of the tokens generated using LIME. This transformer factor corresponds to the concept: “repetitive pattern detector”. In other words, repetitive text sequences will trigger high activation of Φ :,297 . Figure 8: Visualization ofΦ :,322 . This transformer fac- tor corresponds to the concept: “some born in some year” in biography. All of the high-activation contexts contain the beginning of a biography. As shown in the figure, the attributes of someone, name, age, career, and familial relation all have high saliency weights. 4 Discussion Dictionary learning has been successfully used to visualize the classical word embeddings (Arora et al., 2018; Zhang et al., 2019). In this paper, we propose to use this simple method to visual- ize the representation learned in transformer net- works to supplement the implicit “probing-tasks” methods. Our results show that the learned trans- former factors are relatively reliable and can even provide many surprising insights into the linguis- tic structures. This simple tool can open up the transformer networks and show the hierarchical semantic or syntactic representation learned at dif- ferent stages. In short, we find word-level disam- biguation, sentence-level pattern formation, and long-range dependency. The idea of a neural net- work learns low-level features in early layers, and abstract concepts in the later stages are very simi- lar to the visualization in CNN (Zeiler and Fergus, 2014). Dictionary learning can be a convenient tool to help visualize a broad category of neural networks with skip connections, like ResNet (He et al., 2016), ViT models (Dosovitskiy et al., 2020), etc. For more interested readers, we provide an interactive website 1 for the readers to gain some further insights. Acknowledgements We thank our reviewers for their detailed and in- sightful comments. We also thank Yuhao Zhang for his suggestions during the preparation of this paper. References Pretrained bert base model (12 layers).https: //huggingface.co/bert-base-uncased, 1 https://transformervis.github.io/transformervis/ last accessed on 03/11/2021. Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. 2018. Linear algebraic struc- ture of word senses, with applications to polysemy. Transactions of the Association for Computational Linguistics, 6:483–495. Amir Beck and Marc Teboulle. 2009. A fast itera- tive shrinkage-thresholding algorithm for linear in- verse problems.SIAM journal on imaging sciences, 2(1):183–202. Hila Chefer, Shir Gur, and Lior Wolf. 2020. Trans- former interpretability beyond attention visualiza- tion.CoRR, abs/2012.09838. Hila Chefer, Shir Gur, and Lior Wolf. 2021. Generic attention-model explainability for interpreting bi- modal and encoder-decoder transformers.CoRR, abs/2103.15679. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language under- standing.CoRR, abs/1810.04805. AlexeyDosovitskiy,LucasBeyer,Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization.Journal of Machine Learning Research, 12(Jul):2121–2159. Kawin Ethayarajh. 2019. How contextual are contex- tualized word representations? comparing the geom- etry of bert, elmo, and GPT-2 embeddings. InPro- ceedings of the 2019 Conference on Empirical Meth- ods in Natural Language Processing and the 9th In- ternational Joint Conference on Natural Language Processing, EMNLP-IJCNLP, pages 55–65. Associ- ation for Computational Linguistics. Manaal Faruqui, Yulia Tsvetkov, Dani Yogatama, Chris Dyer, and Noah A. Smith. 2015. Sparse overcom- plete word vector representations. InProceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computa- tional Linguistics. W. N. Francis and H. Kucera. 1979. Brown corpus manual. Technical report, Department of Linguis- tics, Brown University, Providence, Rhode Island, US. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recog- nition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 770– 778. John Hewitt and Christopher D. Manning. 2019. A structural probe for finding syntax in word represen- tations. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume 1 (Long and Short Papers). Geoffrey Hinton. 2021. How to represent part-whole hierarchies in a neural network.arXiv preprint arXiv:2102.12627. Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. 2020. How can we know what language models know.Trans. Assoc. Comput. Linguistics, 8:423–438. Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, and Noah A. Smith. 2019. Lin- guistic knowledge and transferability of contextual representations. InProceedings of the 2019 Confer- ence of the North American Chapter of the Associ- ation for Computational Linguistics: Human Lan- guage Technologies, Volume 1 (Long and Short Pa- pers). Association for Computational Linguistics. Emily Reif, Ann Yuan, Martin Wattenberg, Fernanda B. Viégas, Andy Coenen, Adam Pearce, and Been Kim. 2019. Visualizing and measuring the geometry of BERT. InAdvances in Neural Information Process- ing Systems 32: Annual Conference on Neural Infor- mation Processing Systems, (NeurIPS), pages 8592– 8600. Marco Túlio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016."why should I trust you?": Ex- plaining the predictions of any classifier.CoRR, abs/1602.04938. Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020. A primer in bertology: What we know about how BERT works.Trans. Assoc. Comput. Linguis- tics, 8:842–866. Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. 2017. Dynamic routing between capsules.arXiv preprint arXiv:1710.09829. Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R Bowman, Dipan- jan Das, et al. 2019.What do you learn from context?probing for sentence structure in con- textualized word representations.arXiv preprint arXiv:1905.06316. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need.arXiv preprint arXiv:1706.03762. Matthew D Zeiler and Rob Fergus. 2014. Visualizing and understanding convolutional networks. InEuro- pean conference on computer vision, pages 818–833. Springer. Juexiao Zhang, Yubei Chen, Brian Cheung, and Bruno A Olshausen. 2019.Word embedding vi- sualization via dictionary learning.arXiv preprint arXiv:1910.03833. Supplementary Materials A Importance Score (IS) Curves (a)(b) Figure 9: (a) Importance score of 16 transformer fac- tors corresponding to low level information. (b) Im- portance score of 16 transformer factors corresponds to mid level information respectively. The importance score curve’s characteristic has a strong correspondence to a transformer factor’s categorization. Based on the location of the peak of an IS curve, we can classify a transformer factor as low-level, mid-level or high-level. The importance score for low-level transformer factors peak in early layers and slowly decrease across the rest of the layers. On the other hand, the importance score for mid-level and high-level transformers slowly increases and peaks at higher layers. In Figure 9, we show two sets of the examples to demonstrate the clear distinction between those two types of IS curves. Taking a step back, we can also plot IS curve for each dimension of word vector (without sparse cod- ing) at different layers. They do not show any spe- cific patterns, as shown in Figure 10. This makes intuitive sense since we mentioned that each of the entries of a contextualized word embedding does not correspond to any clear semantic meaning. (a)(b) Figure 10: (a) Importance score calculated using cer- tain dimension of word vectors without sparse coding. (b) Importance score calculated using sparse coding of word vectors. B LIME: Local Interpretable Model-Agnostic Explanations After we trained the dictionaryΦthrough non- negative sparse coding, the inference of the sparse code of a given input is α(x) = arg min α∈R ||x−Φα|| 2 2 +λ||α|| 1 For a given sentence and index pair(s,i), the em- bedding of wordw=s[i]by layerlof transformer isx (l) (x,i). Then we can abstract the inference of a specific entry of sparse code of the word vector as a black-box scalar-value functionf: f((s,i)) =α(x (l) (s,i)) LetRandomMaskdenotes the operation that generates perturbed version of our sentencesby masking word at random location with “[UNK]” (unkown) tokens. For example, a masked sentence could be [Today is a [‘UNK’],day] Lethdenote a encoder for perturbed sentences compared to the unperturbed sentences, such that h(s) t = 0ifs[t] =[‘UNK’] 1Otherwis The LIME algorithm we used to generated saliency map for each sentences is the following: Algorithm 1Explaining Sparse Coding Activation using LIME Algorithm 1:S=h(s) 2:Y=f(s) 3:foreachiin1,2,...,Ndo 4:s ′ i ←RandomMask(s) 5:S ←S∪h(s ′ i ) 6:Y←Y∪f(s ′ i ) 7:end for 8:w←Ridge w (S,Y) WhereRidge w is a weighted ridge regression defined as: w= arg min w∈R t ||SΩw−Y|| 2 +λ||w|| 2 2 Ω =diag(d(h(S 1 ), ~ 1),d(h(S 2 ), ~ 1),·,d(h(S n ), ~ 1)) d(·,·) can be any metric that measures how much a perturbed sentence is different from the original sentence. If a sentence is perturbed such that every token is being masked, then the distanceh(h(s ′ ), ~ 1) should be 0, if a sentence is not perturbed at all, thenh(h(s ′ ), ~ 1)should be 1. We choosed(·,·)to be cosine similarity in our implementation. In practice, we also uses feature selection. This is done by running LIME twice. After we obtain the regression weightw 1 for the time, we use it to find the firstkindices corresponds to the entry in w 1 with highest absolute value. We use thosek index as location in the sentence and apply LIME for the second time with only those selected indices from step 1. Overall, the regression weightwcan be regarded as a saliency map. The higher the weightw k is, the more important the words[k]in the sentence since it contributes more to the activation of a specific transformer factor. We could also have negative weight inw. In general, negative weights are hard to interpret in the context of transformer factor. The activation will increase if they are removed those word correspond to negative weights. Since a transformer factor corresponds to a specific pattern, then word with negative weights are those word in a context that behaves “opposite" of this pattern. C The Details of the Non-negative Sparse Coding Optimization LetSbe the set of all sequences, recall how we de- fined word embedding using hidden state of trans- former in the main section:X (l) =x (l) (s,i)|s∈ S,i∈[0,len(s)]as the set of all word embedding at layerl, then the set of word embedding across all layers is defined as X=X (1) ∪X (2) ∪·∪X (L) In practice, we use BERT base model as our trans- former model, each word embedding vector (hid- den state of BERT) is dimension 768. To learn the transformer factors, we concatenate all word vector x∈Xinto a data matrixA. We also definedf(x) to be the frequency of the token that is embedded in word vectorx. For example, ifxis the embed- ding of the word “the”, it will have a much larger frequency i.e.f(x)is high. Usingf(x), we define the Inverse Frequency MatrixΩ:Ωis a diagonal matrix where each entry on the diagonal is the square inverse frequency of each word, i.e. Ω =diag( 1 √ f(x 1 ) , 1 √ f(x 2 ) ,...) Then we use a typical iterative optimization pro- cedure to learn the dictionaryΦdescribed in the main section: min A 1 2 ‖X−ΦA‖ 2 F +λ ∑ i ‖α i ‖ 1 ,s.t.α i 0,(2) min Φ 1 2 ‖X−ΦΩA‖ 2 F ,‖Φ :,j ‖ 2 ≤1.(3) These two optimizations are both convex, we solve them iteratively to learn the transformer fac- tors: In practice, we use minibatches contains 200 word vectors asX. The motivation of apply In- verse Frequency MatrixΩis that we want to make sure all words in our vocabulary has the same con- tribution. When we sample our minibatch fromA, frequent words like “the” and “a” are much likely to appear, which should receive lower weight dur- ing update. Optimization 2 can converge in 1000 steps using the FISTA algorithm 2 . We experimented with dif- ferentλvalues from 0.03 to 3, and chooseλ= 0.27 to give results presented in this paper. Once the sparse coefficients have been inferred, we update our dictionaryΦbased on Optimization 3 by one step using an approximate second-order method, where the Hessian is approximated by its diagonal to achieve an efficient inverse (Duchi et al., 2011). The second-order parameter update method usually leads to much faster convergence. Empirically, we train 200k steps and it takes roughly 2 days on a Nvidia 1080 Ti GPU. D Hyperlinks for More Transformer Visualization In the following three sections, we provide visu- alization of more example transformer factor in low-level, mid-level, and high-level. Here’s table of Contents that contain hyperlinks which direct to each level: • Low-Level: E • Mid-Level: F • High-Level: G 2 The FISTA algorithm can usually converge within 300 steps, we use 1000 steps nevertheless to avoid any potential numerical issue. E Low-Level Transformer Factors Transformer factor 2 in layer 4 Explaination: Mind: noun, the element of a person that enables them to be aware of the world and their experiences. • that snare shot sounded like somebody’ d kicked open the door to your mind". •i became very frustrated with that and finally made up my mind to start getting back into things." •when evita asked for more time so she could make up her mind, the crowd demanded," ¡ ahora, evita,< •song and watch it evolve in front of us... almost as a memory in your head. •was to be objective and to let the viewer make up his or her own mind." •managed to give me goosebumps, and those moments have remained on my mind for weeks afterward." • rests the tir’ d mind, and waking loves to dream • , tracks like’ halftime’ and the laid back’ one time 4 your mind’ demonstrated a[ high] level of technical precision and rhetorical dexter • so i went to bed with that on my mind". • ment to a seed of doubt that had been playing on mulder’ s mind for the entire season". • my poor friend smart shewed the disturbance of his mind, by falling upon his knees, and saying his prayers in the street •donoghue complained that lessing has not made up her mind on whether her characters are" the salt of the earth or its sc •release of the new lanois@-@ produced album, time out of mind. • sympathetic man to illegally" ghost@-@ hack" his wife’ s mind to find his daughter. •this album veered into" the corridors" of flying lotus’" own mind", interpreting his guest vocalists as" disembodied phantom Transformer factor 16 in layer 4 Explaination: Park: noun, ’park’ as the name • allmusic writer william ruhlmann said that" linkin park sounds like a johnny@-@ come@-@ lately to an •nington joined the five members xero and the band was renamed to linkin park. • times about his feelings about gordon, and the price family even sat away from park’ s supporters during the trial itself. •on 25 january 2010, the morning of park’ s 66th birthday, he was found hanged and unconscious in his • was her, and knew who had done it", expressing his conviction of park’ s guilt. •jeremy park wrote to the north@-@ west evening mail to confirm that he • vanessa fisher, park’ s adoptive daughter, appeared as a witness for the prosecution at the •they played at< unk> for years before joining oldham athletic at boundary park until 2010 when they moved to oldham borough’ s previous ground,< •theme park guests may use the hogwarts express to travel between hogsmead • s strength in both singing and rapping while comparing the sound to linkin park. •in a statement shortly after park’ s guilty verdict, he said he had" no doubt" that • june 2013, which saw the band travel to rock am ring and rock im park as headline act, the song was moved to the middle of the set •after spending the first decade of her life at the central park zoo, pattycake moved permanently to the bronx zoo in 1982. •south park spoofed the show and its hosts in the episode" south park is gay!" • harrison" sounds like he’ s recorded his vocal track in one of the park’ s legendary caves". Transformer factor 30 in layer 4 Explaination: left: verb, leaving, exiting •did succeed in getting the naval officers into his house, and the mob eventually left. • all of the federal troops had left at this point, except totten who had stayed behind to listen to • saying that he has left the outsiders, kovu asks simba to let him join his pride •eventually, all boycott’ s employees left, forcing him to run the estate without help. •the story concerned the attempts of a scientist to photograph the soul as it left the body. •in time and will slowly improve until he returns to the point at which he left. •peggy’ s exit was a" non event", as" peggy just left, nonsensically and at complete odds with everything we’ ve •over the course of the group’ s existence, several hundred people joined and left. •no profit was made in six years, and the church left, losing their investment. •on 7 november he left, missing the bolshevik revolution, which began on that day. •he had not re@-@ written his will and when produced still left everything to his son lunalilo. •they continued filming as normal, and when lynch yelled cut, the townspeople had left. •with land of black gold( 1950), a story that he had previously left unfinished, instead. •he was infuriated that the government had left thousands unemployed by closing down casinos and brothels. •an impending marriage between her and albert interfered with their studies, the two brothers left on 28 august 1837 at the close of the term to travel around europe Transformer factor 33 in layer 4 Explaination: light: noun, the natural agent that stimulates sight and makes things visible: •forced to visit the sarajevo television sta- tion at night and to film with as little light as possible to avoid the attention of snipers and bombers. • by the modest, cream@-@ colored attire in the airy, light@-@ filled clip. •the man asked her to help him carry the case to his car, a light@-@ brown volkswagen beetle. • they are portrayed in a particularly sympathetic light when they are killed during the ending. •caught up" was directed by mr. x, who was behind the laser light treatment of usher’ s 2004 video" yeah!" • piracy in the indian ocean, and the script depicted the pirates in a sympathetic light. • without the benefit of moon light, the light horsemen had fired at the flashes of the enemy’ s •second innings, voce repeated the tactic late in the day, in fading light against woodfull and bill brown. •, and the workers were transferred on 7 july to another facility belonging to early light, 30 km away in< unk> town. •unk> brooklyn avenue ne near the university of washington campus in a small light@-@ industrial building leased from the university. •factory where the incident took place is the< unk>(" early light") toy factory(< unk>), owned by hong •, a 1934 comedy in which samuel was portrayed in an unflattering light, and mrs beeton, a 1937 documentary,< unk> •stage effects and blue@-@ red light transitions give the video a surreal feel, while a stoic crowd make • set against the backdrop of mumbai’ s red@-@ light districts, it follows the travails of its personnel and principal, •themselves on the chinese flank in the foothills, before scaling the position at first light. Transformer factor 47 in layer 4 Explaination: plants: noun, vegetation • the distinct feature of the main campus is the mall, which is a large tree – laden grassy area where many students go to relax. • each school in the london borough of hillingdon was invited to plant a tree, and the station comman- der of raf northolt, group captain tim o • its diet in summer contains a high proportion of insects, while more plant items are eaten in autumn. •large fruitings of the fungus are often associated with damage to the host tree, such as that which occurs with burning. •she nests on the ground under the cover of plants or in cavities such as hollow tree trunks. • orchards, heaths and hedgerows, especially where there are some old trees. •the scent of plants such as yarrow acts as an olfactory attractant to females. • of its grasshopper host, causing it to climb to the top of a plant and cling to the stem as it dies. •well@-@ drained or sandy soil, often in the partial shade of trees. •food is taken from the ground, low@-@ growing plants and from inside grass tussocks; the crake may search leaf •into his thought that the power of gravity( which brought an apple from a tree to the ground) was not limited to a certain distance from earth, •they eat both seeds and green plant parts and consume a variety of animals, including insects, crustaceans • fyne, argyll in the 1870s was named as the uk ’ s tallest tree in 2011. • ", or colourless enamel, as in the ground areas, rocks and trees. •produced from 16 to 139 weeks after a forest fire in areas with coniferous trees. This is the end of visualization of low-level transformer factor. Click [D] to go back. F Mid-Level Transformer Factors Transformer factor 13 in layer 10 Explaination: Unit exchange with parentheses: e.g. 10 m (1000cm) • 14@-@ 16 hand( 56 to 64 inches( 140 to 160 cm)) war horse is that it was a matter of pride to a •, behind many successful developments, defaulted on the$ 214 million($ 47 billion) in bonds held by 60@,@ 000 investors; the van •straus, behind many successful developments, defaulted on the$ 214 million($ 47 billion) in bonds held by 60@,@ 000 investors; • the niche is 4 m( 13 ft) wide and 3@. • with a top speed of nearly 21 knots( 39 km/ h; 24 mph). •@ 4 billion( us$ 21 million) — india’ s highest@- @ earning film of the year •) at deep load as built, with a length of 310 ft( 94 m), a beam of 73 feet 7 inches( 22@. •@ 3@-@ inch( 160 m) calibre steel barrel. •and gave a maximum speed of 23 knots( 43 km/ h; 26 mph). • 2 km) in length, with a depth around 790 yards( 720 m), and in places only a few yards separated the two sides. • hull provided a combined thickness of between 24 and 28 inches( 60 – 70 cm), increasing to around 48 inches( 1@. •switzerland, austria and germany; and his mother, lynette federer( born durand), from kempton park, gauteng, is •@ 2 in( 361 m) thick sides. •) and a top speed of 30 knots( 56 km/ h; 35 mph). •, an outdoor seating area( 4@,@ 300 square feet( 400 m2)) and a 2@,@ 500@-@ square@ Transformer factor 24 in layer 10 Explaination: Male name •divorcing doqui in 1978, michelle married robert h. tucker, jr. the following year, changed her name to gee tucker, moved back •divorced doqui in 1978 and married new orleans politician robert h. tucker, jr. the following year; she changed her name to gee tucker and became •including isabel sanford, when chuck and new orleans politician robert h. tucker, jr. visited michelle at her hotel. • of 32 basidiomycete mushrooms showed that mutinus elegans was the only species to show antibiotic( both antibacterial •amphicoelias, it is probably synonymous with camarasaurus grandis rather than c. supremus because it was found lower in the •[ her] for warmth and virtue" and mehul s. thakkar of the deccan chronicle wrote that she was successful in" deliver[ • em( queen latifah) and uncle henry( david alan grier) own a diner, to which dorothy works for room and board. •in melbourne on 10 august 1895, presented by dion boucicault, jr. and robert brough, and the play was an immediate success. •in the early 1980s, james r. tindall, sr. purchased the building, the construction of which his father had originally financed •in 1937, when chakravarthi rajagopalachari became the chief minister of madras presidency, he introduced hindi as a compulsory • in 1905 william lewis moody, jr. and isaac h. kempner, members of two of galveston’ • also, walter b. jones, jr. of north carolina sent a letter to the republican conference chairwoman cathy •empire’ s leading generals, nikephoros bryennios the elder, the doux of dyrrhachium in the western balkans • in bengali as< unk>: the warrior by raj chakra- borty with dev and mimi chakraborty portraying the lead roles. •on 1 june 1989, erik g. braathen, son of bjørn g., took over as ceo Transformer factor 25 in layer 10 Explaination: Attributive Clauses • which allows japan to mount an assault on the us; or kill him, which lets the us discover japan’ s role in rigging american elections — •certain stages of development, and constitutive heterochromatin that consists of chromosome structural components such as telomeres and centromeres •to the mouth of the nueces river, and oso bay, which extends south to the mouth of oso creek. •@,@ 082 metric tons, and argentina, which ranks 17th, with 326@,@ 900 metric tons. • of$ 42@,@ 693 and females had a median income of$ 34@,@ 795. • ultimately scored 14 points with 70 per cent shooting, and crispin, who scored twelve points with 67 per cent shooting. •and is operated by danish air transport, and one jetstream 32, which seats 19 and is operated by helitrans. •acute stage, which occurs shortly after an initial infection, and a chronic stage that develops over many years. •, earl of warwick and then william of lancaster, and ada de warenne who married henry, earl of huntingdon. • who ultimately scored 14 points with 70 per cent shooting, and crispin, who scored twelve points with 67 per cent shooting. •in america, while" halo/ walking on sunshine" charted at number 4 in ireland, 9 in the uk, 10 in australia, 28 in canada •five events, heptathlon consisting of seven events, and decathlon consisting of ten< unk> every multi event, athletes participate in a • @-@ life of 154@,@ 000 years, and 235np with a half@-@ life of 396@. •comfort, and intended to function as the prison, and the second floor was better finished, with a hall and a chamber, and probably operated as the •b, which serves the quonset freeway, and exit 7a, which serves route 402( frenchtown road), another spur route connecting the Transformer factor 42 in layer 10 Explaination: Some kind of disaster, something unfortunate happened •after the first five games, all losses, jeff carter suffered a broken foot that kept him out of the line@-@ up for •allingham died of natural causes in his sleep at 3: 10 am on 18 july 2009 at his •upon reaching corfu, thousands of serb troops began showing symptoms of typhus and had to be quarantined on the island of< un •than a year after the senate general election, the september 11, 2001 terrorist attacks took place, with giuliani still mayor. •the starting job because fourth@-@ year junior grady was under suspension related to driving while intoxicated charges. • his majesty, but as soon as they were on board ship, they died of melancholy, having refused to eat or drink. •on 16 september 1918, before she had even gone into action, she suffered a large fire in one of her 6@-@ inch magazines, and • orange goalkeeper for long@-@ time starter john galloway who was sick with the flu. •in 1666 his andover home was destroyed by fire, supposedly because of" the carelessness of the maid". •the government, on 8 february, admitted that the outbreak may have been caused by semi@-@ processed turkey meat imported directly • ikromo came under investigation by the justice office of the dutch east indies for publishing several further anti@-@ dutch editorials. •that he could attend to the duties of his office, but fell ill with a fever in august 1823 and died in office on september 1. •@ 2 billion initiative to combat cholera and the construction of a$ 17 million teaching hospital in< unk • he would not hear from his daughter until she was convicted of stealing from playwright george axelrod in 1968, by which time rosaleen • relatively hidden location and proximity to piccadilly circus, the street suffers from crime, which has led to westminster city council gating off the man in Transformer factor 50 in layer 10 Explaination: Doing something again, or making something new again •2007 saw the show undergo a revamp, which included a switch to recording in hdtv, the introduction • during the ship’ s 1930 reconstruction; the maxi- mum elevation of the main guns was increased to+ 43 degrees, increasing their maximum range from 25@, •hurricane pack 1 was a revamped version of story mode; team ninja tweaked the •she was fitted with new engines and more powerful water@-@ tube boilers rated at 6@ •from 1988 to 2000, the two western towers were substantially overhauled with a viewing platform provided at the top of the north tower. •latest missoula downtown master plan in 2009, increased emphasis was directed toward redeveloping the north side’ s former rail yard and the area •1896: the ribbon of the army version medal of honor was redesigned with all stripes being vertical. • the new badge includes a star to represent the european cup win in 1982, and •missoula downtown master plan in 2009, in- creased emphasis was directed toward redeveloping the north side’ s former rail yard and the area just •also assisted in comprehensive infrastructure renovations, restored a dependable supply of electricity, revamped the baggage handling facilities as well as the arrival and departure lounge •hurricane pack 1 was a revamped version of story mode; team ninja tweaked the encounters •1896: the ribbon of the army version medal of honor was redesigned with all stripes being vertical. •from 1988 to 2000, the two western towers were substantially overhauled with a viewing platform provided at the top of the north tower • assisted in comprehensive infrastructure renova- tions, restored a dependable supply of electricity, revamped the baggage handling facilities as well as the arrival and departure lounges •bond series and the fourth to star roger moore as bond; the plot was significantly changed from the novel to include excursions into space. Transformer factor 51 in layer 10 Explaination: apostrophe s, possesive •the irish times was critical of the book’ s text but wrote positively of the included pho- tographs. • if it survived long enough to become old@-@ fashioned it was likely to be •you by phil spector as his inspirations, which resulted to the album’ s wall of sound resonance. •the irish times was critical of the book’ s text but wrote positively of the included photographs. • album to the wu tang clan and nine inch nails, particularly comparing the album’ s production( which was done by various producers with executive producer don gilmore •to the wu tang clan and nine inch nails, particu- larly comparing the album’ s production( which was done by various producers with executive producer don gilmore) •toward the commoners and interested in easing their burden but suspicious about the letter’ s true purpose, reluctantly signed the document under intense pressure from the french •the novel’ s reception was even warmer than that of its predecessor; waugh was •first song selected for inclusion after her mother’ s recommendation and the song’ s melancholic lyrics. • it divided critics at the time; although they praised the game’ s writing and scale of choice, they criticized its technical flaws. •mgm executive al lewin said that several years after the film’ s release stroheim asked him for the cut footage. •the game’ s production was turbulent, as the design’ s scope exceeded the available resources • nicki escudero from the phoenix new times noted the song’ s superficial themes which included lyrics about" sex, money and cheating" •mgm executive al lewin said that several years after the film’ s release stroheim asked him for the cut footage. • labrie said that there was" a lot of discussion" about the song’ s wording and how direct it should be. Transformer factor 86 in layer 10 Explaination: Pattern: Consecutive years, this is convention to name foodball/rugby game season • with york the previous season, signed a contract until the end of 2013 – 14 and sheffield united midfielder elliott whitehouse signed on a one@-@ • as of the end of the 2014 – 15 season, aston villa have spent 104 seasons in the top tier of english •won 13 and drew two of their opening 15 league matches of the 1985 – 86 campaign, and seemed destined to win the first division title. •mcallister, still without a goal in 2009 – 10, couldn’ t get on the scoresheet in the three games •john bentley led united to a fourth@-@ place finish in 1912 – 13. •he made 46 appearances, scoring three goals, in the 2001 – 02 season before spending the close season with the kalamazoo kingdom in the •he moved to basingstoke town towards the end of 2001 – 02, making his debut in march 2002. •7[ note 1] was the worst record in the nhl for 2011 – 12 and the first time in franchise history they finished in last place. •side, who withdrew from the football league at the end of the 1893 – 94 season after finishing bottom of the second division. •spent a year as a physics instructor at the university of minnesota in 1916 – 17, then two years as a research engineer with the westinghouse lamp •defeat was a 7 – 2 loss to witton albion in the 2001 – 02 season. •york achieved three successive wins for the first time in 2013 – 14 after beating northampton 2 – 0 away, with bowman and fletcher scoring in •he started to develop more of an offensive game, finishing off the 2001 – 02 season with 58 points in the 47 games he played in seattle. •suart limited matthews to 19 league appearances in 1958 – 59. •jaw warriors of the western hockey league( whl) during the 2000 – 01 season. Transformer factor 99 in layer 10 Explaination: past tense •r.in their review of rihanna’ s top 20 songs, time out ranked" man down" as their tenth best track, writing that it is •rolling stone ranked" imagine" number three on its list of" the 500 greatest songs •japan’ s computer entertainment rating organiza- tion( cero) rated ninja gaiden and black, on their release, as 18+ •ultimate classic rock ranked" lola" as the kinks’ third best song, saying" • adrien begrand of popmatters described" south of heaven" as" an unorthodox set opener •columbia records released it as the album’ s fourth and final single on june 14, •rolling stone ranked it the best song of 2009 and the 36th@-@ best song • indielondon’ s jack foley noted" wind it up" as a highlight of the sweet escape and called •premiere magazine listed frank booth, played by dennis hopper, as the fifty@-@ •columbia records released" crazy in love" on may 18, 2003, as the lead •the times considered the production the best since the original, and praised it for its fidelity •the good food guide ranked hibiscus as the eighth@-@ best restaurant in the uk •viz media later began releasing the manga as simply" ral grad" in february 2008. •entertainment weekly magazine ranked" crazy in love" forty@-@ seven in its list of •the japanese publisher nihon bungeisha released the series in collected volumes from january 2000 to september 2009. Transformer factor 102 in layer 10 Explaination: African name • s 1966 to 1971 live performances in paris, prepared to press the album once mwanga provided the label with the record< unk>. • of america" with the nhk symphony orchestra, but cancelled both deals upon mwanga’ s return from japan. •1966 to 1971 live performances in paris, prepared to press the album once mwanga provided the label with the record< unk>. • and langston hughes, and by modern african poets and folk artists such as kwesi brew and efua sutherland, which also influenced her auto •america" with the nhk symphony orchestra, but cancelled both deals upon mwanga’ s return from japan. • du bois was buried in accra near his home, which is now the du bois memorial centre. •du bois returned to africa in late 1960 to attend the inauguration of nnamdi azikiwe as the first african governor of nigeria. •david mcgurk, lanre oyebanjo, danny parslow, tom platt and chris smith signed new • and moderate nationalist parties, the most prominent of which was bishop abel muzorewa’ s united african national council( uanc). •a few weeks after of human feelings was recorded, mwanga went to japan to negotiate a deal with trio records to have the •and was part of two large campaigns, one to witu and another to mwele. •returned to africa in late 1960 to attend the inauguration of nnamdi azikiwe as the first african governor of nigeria. •in april, mwanga arranged another session at cbs studios in new york city, and coleman •the government and moderate nationalist parties, the most prominent of which was bishop abel muzorewa’ s united african national council( uanc). •ralambo’ s father, andriamanelo, had established rules of succession by which ralambo’ Transformer factor 125 in layer 10 Explaination: Describing someone in a para- phrasing style. Name, Career • journalist tim judah suggests that the move may have been motivated by a desire to control a •the historian nora berend says that the latter measure" may have adversely affected •from the pyx that were not assayed, and nu- mismatic historian roger burdette speculates that ashbrook, generally well@- • the pyx that were not assayed, and numismatic historian roger burdette speculates that ashbrook, generally well@-@ •the cricket historian derek birley notes that many of these bowlers imitated the methods of • critic roberta reeder notes that the early poems always attracted large numbers of admirers •; the figures for the last two years are not available, but sf historian mike ashley estimates that fantastic paid circulation may have been as low as 13@ •aesthetically, ign’ s tal blevins noted that the game had" a very distinct 40s • sf historian everett bleiler notes that hersey did not mention the venture in his •similarly, duke university professor, mark anthony neal, writes, “ nas was at the forefront of a renaissance •club reviewer erik adams wrote that the episode was a perfect mix, between the more subtle • commenting on the album and its use of samples, pitchfork’ s jeff weiss claims that both nas and his producers found inspiration for the album’ •the historian stanley karnow said of ky and thi:" both fl •that were not assayed, and numismatic historian roger burdette speculates that ashbrook, generally well@-@ treated by the •irataba was described as an eloquent speaker, and linguist leanne hinton suggests that he was among the first mohave people to Transformer factor 134 in layer 10 Explaination: Transition sentence •fanny workman have tended to slight or belittle her achievements, but contemporaries, unaware of the far greater accomplishments to come, held the workmans • scheduled to air in its regular half@-@ hour time slot, but nbc later announced it would be expanded to fill an hour time slot beginning a •wine and savoy cabbage with a red wine and smoked chocolate sauce, but he otherwise felt that the food was" over@-@ worked" and the •lap melee when he was hit by romain grosjean; webber was forced to pit straight away, while grosjean was given a ten@ •ra. one was initially scheduled to release on 3 june 2011, but delays due to a lengthy post@-@ production process and escalating • yamina nomads who were centered at tuttul, and the rebels were supported by yamhad’ s king sumu@-@ •the item was intended simply as a piece of news, but telegraph lines quickly spread the news throughout the state, fueling procession sentiment. •both twc and comcast began trials of services based on the system; turner broadcasting was an early supporter of the system, providing access to tbs and •k> have claimed that he proposed a dictatorship for robespierre, but nonetheless some of them considered him to be redeemable, or at least •@ 2 style with superfiring pairs of turrets fore and aft; the middle turrets were not superfiring, and had a funnel between them. • romani being ordered to move out with supplies for the advancing troops, but 150 men, most of whom were past the end of their contracts and entitled to • ’ s boats to enter the creek into which the schooner had fled, the small craft entering the waterway in the hope of storming and capturing the vessel • -@ person shooter elements and a unique on rails control scheme, but the core adventure@-@ style gameplay has been compared to myst and snatch • stanza 6; movement 4 incorporates ideas from stanzas 7 – 14, and movement 5 relies on stanzas 15 and< unk> movement 2, •as corps troops that were usually allocated at a rate of one per division; several of the militia units were also later designated australian imperial force units, after Transformer factor 152 in layer 10 Explaination: in some locations •while most breeding stallions and racehorses of the era had stable companions, waxy reportedly was fond of rabbits in his later years and •planet and the helter skelter music bookshop have also been based on the street. • the central bank of somalia, the national monetary authority, also has its headquarters in mogadishu. •some allotropes of the other actinides also exhibit similar behaviour, though to a lesser degree. • ; fortune 1000 technology company< unk>, for instance, is headquartered in the area. •musical@-@ comedy television series maid marian and her merry men were filmed in cleeve abbey. •ireland,< unk> and donegal bay in particular, have popular surfing beaches, being fully exposed to the atlantic ocean. • lstoy’ s war and peace and chekhov’ s peasants both feature scenes in which wolves are hunted with hounds and< unk>. •while most breeding stallions and racehorses of the era had stable companions, waxy reportedly was fond of rabbits in his later years and" •the lancashire and england test cricketer paul allott was born in altrincham. • asura, the demon devotee of shiva, are both credited with building temples or cut caves to live. •forbidden planet and the helter skelter music bookshop have also been based on the street. • thopedic shriners hospitals in the u. s. is also located in spokane. •dykes to watch out for and fun home, was born in lock haven in 1960. • < unk>, and alessandra ambrosio have each worn two fantasy bras. This is the end of visualization of mid level transformer factor. Click [D] to go back. G High-Level Transformer Factors Transformer factor 297 in layer 10 with saliency map Explaination: repetitive structure detector frontierworks, and an original soundtrack by avex group were created based on the game. dramacd: tales of graces1 to 4 are side stories that take place during the game’ s plot. they were released between may 26, 2010 and august 25, 2010.anthology drama cd:talesof graces f 2010winter,anthologydrama cd:tales of graces f2011summer, anthology drama cd:talesof graces f 2012 winter, anthology drama cd:talesof graces f 2012 summer, anthologydramacd:talesofgracesf 2013winter, and anthology dramacd: tales of gracesf 2013 cobrand platinum cardholders, and citibank eva aircobrandworld card) the infinity( infinity mileagelands diamond, royal laurel/premium laurel classpassengers,staralliancefirst/ businessclass passengers, americanexpress centurion/ eva air cobrand platinum cardhold- ers,and citibankeva air cobrand world cardholders) the star( infinity mileagelands diamond/ gold, royal laurel/ premiumlaurelclass passengers,staralliancefirst/business classpassengers,staralliancegold members, americanexpresscenturion/ eva air cobrand platinum cardholders,citibank eva air cobrand world cardholders, business customers, quicklyset online< unk> alight"."can’ tgetyououtofmyhead" was chosen as the lead single from minogue’seighth studio album fever, and it was released on 8 september 2001 byparlophone in australia,whileinthe united kingdomandother european countries it was released on 17 september."can’t getyououtofmyhead" was w ritten and produced bycathydennisandrob davis, who had beenputtogether by british artist manager simon fuller,who wanted the duo to come up withasong for british pop group s club 7. the song was recorded using cuba typhoonstatus with two@-@minute sustained winds estimated at 125 km/ h( 78 mph). around 1700 utconmay 31, the storm tracked approximately 65 km( 40 mi) west of iwojima.roughly five hours later,itmoved within 15 km( 10 mi) ofchichi@-@jima whereapressureof 992 mb( hpa; 29@.@ 30 inhg)wasmeasured.sustainedwindson chichi@-@jimareached 95 km/ h(60 mph);however, these were determined to be unrepresentative of lucille’ s actual intensity due firstbookinvocal music.themodern musicseries. book 1. new york, new york: silver burdette and company. smith, eleanor( 1901). a second book in vocal music.themodern musicseries. book 2. new york, new york: silver burdette and company. smith, eleanor(1901). a third book in vocal music.themodernmusic series. book 3.newyork, new york: silver burdette and company. smith,eleanor( 1905).a fourthbookinvocalmusic.themodern musicseries.book4.new york,new york:silver burde @breaking eight weeks at number one on the airplay chart ofthecountry andbecame the first togarner3000 radio plays in a single week. subsequently,it becamethemost@-@ played song of 2001 in the region."can’ tgetyououtofmyhead" was certified platinumbythe british phonographic industry for shipments of 600@,@ 000 units in 2001. the certificationwasupgraded to double@-@ platinum in 2015, denoting shipmentsof1@,@ 200@,@ 000 units. in the united states,"can’ tgetyououtofmyhead"peaked at number seven on the chart.in mid@-@ august 2015,"la mordidita" earned martin his twenty@- @ sixth top ten hit on hot latin songs. he became the fourth artistwiththemosttop tens in the29@-@ year history of the chart.in late august 2015,martinearnedwith"la mordidita"hisfifteenthnumber@-@ one on the latin airplay chart(up 58 percent, to 11@.@ 8 million audience impressions). eventually,"lamordidita"peakedat number six on the us hot latin songs chart, number one on latin airpla y and ,was delivered to sukhoi’ s experimental workshop to be outfitted with exclusive systems. built by knaapo, its structure has increased carbon@-@ fibre andal@-@ li content. installed wasthe2d thrust@-@vectoringlyulka al@-@ 31fp,aninterimmeasure pending the availability of theal@-@ 37fu(< unk>< unk>,"afterburner@-@ controlled").the 3d thrust@-@ vectoringlyulkaal@-@ 37fuwasstillindevelopment. theal@-@ 31fp, in ke’ s former band, thoughescapethe fate only charted at number 25, seven spots lower thanthedruginmeis you,despite equal sales.initssecondweek on sales,the druginmeisyoudroppedabout 70% in the united states,selling5@,@870 copies. this dropped the album 60 spots to number 79 on the billboard 200, and brought total us sales for the album to around 24@,@ 000 copies. on the billboard charts, thedruginmeis you charted at number twoonthetophard rock albums chart, numberthreeonthetop alternative albums and top rock albums charts, no,no,no",reachednumber one on the billboardhotr& b/ hip@-@ hop singles& tracks and number three on the billboard hot 100. its follow@-@ upsingle," with mepart1" failed to reproducethesuccessof"no, no,no".meanwhile,the group featured on a song fromthesoundtrack album of the romantic drama why do fools fall in love and" get on the bus" had a limited release in europe and other markets.in 1998, destiny’ s child garnered three soul train lady of soul awards including best new artistfor"no,no ,no oistic warmongers.alexanderkrivenko( jonathanadams)finally, introduced in trivial games and paranoid pursuits, is russian alexander krivenko, the commander of the moonbase where the ispf have their headquarters. a winner of the nobel prize for medicine, it iskrivenko’ s research into bone damage that has contributed to enabling humanity to access space easily. although the star cops are independent,spring’ s relationshipwithkrivenko is often deferential and hefrequentlyseemstocapitulateto krivenko’swishes.== production history===== origins= thatbuild faith:fromthelife and min- istry of thomas s.monson,saltlake city, utah:deseretbook, isbn 978@-@0@-@ 87579@-@ 901@-@ 8 —( 1996), faith rewarded:a personal account of prophetic promises to the east german saints, salt lake city,utah:deseretbook, isbn 978@-@ 1@-@ 57345@-@ 186@-@4— —( 1997), invitation to exaltation,salt lake city,utah: deseretbook , isbn 978@- dell, tom( 2015).gunnerkrigg court vol- ume 5:< unk>.gunnerkrigg court.archaia studios press.isbn 978@-@< unk>.=== side comics=== siddell, tom( 2013).annie in the forest partone.beyond thewalls. robotvoicecomics.siddell, tom(2013). annie in theforestpart two. beyond the walls. robotvoicecomics.siddell, tom( 2015). traveller.beyondthewalls.robotvoice comics. === expl anatory footnotes====== 95@.@4kn)f402@-@r@-@< unk> engine, while later examples were fitted with the 23@,@ 000 lbf( 105@.@ 8 kn) f402@-@r@-@408a.inthe early 2000s,17 tav@-@8bs were upgraded to includeanight@-@ attack capability,the f402@-@r@-@ 408engine, and software and structural changes.< unk> in 1991,the night attack harrier was the first upgrade of theav@-@ 8 ,aitrus’ meeting with ti’ ana, and the birth of their son gehn.the book also explains the destruction ofthed’nicivilization.two d’ni, veovisanda’ gaeris, plot to destroy theircivilization, which theybelievehas been corrupted. veovisanda’ gaeris create a plague which wipes outmanyofthed’niand followsthemthroughthe ages.veovis is murdered bya’ gaerisforrefusing towrite an age where the two of them would have been worshipped as gods, and aitrus sacrifices himself in order to inants".a gbrmpabriefingstated the companyhad" threatenedacompensation claim of$< unk>shouldthegbrmpaintendto exert authority over the company’ s operations". in responsetothe< unk> of the dumping inci- dents, thegbrmpastated: wehavestrongly encouraged the company to investigateoptions thatdon’ tentail releasing thematerialto the environment and to develop a management plan to eliminate this potential hazard;however, gbrmpadoes not have legislative control over how the< unk> tailings dam is managed.===< unk>=== following a ofwarpedtour.following this, a lesson in romantics wasreleasedon july 10 through fearless records. in august, the band went on tour with olympiaandsoundthealarm. the music videofor"whenigethome,you’re so dead", directed by marco de la torre, was filmed in september.inlateseptember2007, the band supported paramoreinjapan an d australia.the band wentonaco@-@ headlining tour with madina lake in october and november. the"when igethome,you’ re sodead" musicvideo wasreleasedonnovember 14, and the single was released on ofthe english football league including promo tion and relegation. the player’ steam begins with alowrating in an8@-@ teamleague. by winning games, the player earns credits, which can be used to purchase the contracts of free agents.byfinishinghighin the8@-@teamleague,theplayer’s teamadvancesto a 16@-@ team league and eventually a 32@-@ team league.the player improves their team by periodically signing free agents,as thecompetitionis tougher in each league. the player wins themodeafter winning a playoff tournament in the 32@-@ teamleague odaministra kulturyisztukiii< unk>) 1972 – member of commission" poland 2000" of the polishacademyof sciences 1973 prize of the minister offoreignaffairs for popularizationof polish cultureabroad(< unk> ministra< unk>< unk> za< unk>< unk> kultury za<unk>)literary prize of the minister of culture and art(< unk>< unk> ministra kulturyisztuki)and honorary member of sciencefictionwritersof america1976– stateprize 1st level in the area of topower the antarctic outpost. above earth, ba’ al’ s armada arrives. tothedispleasure of his subordinates, the other system lords,ba’al announcesthat he will treat thetau<unk> leniently. suspicious about ba’al’ s thorough knowledgeofearth, qetesh betrayshimand forces him to tell her everything. she orders the destructionof mcmurdo and the ancient outpost in ba’al’s name,butshekillsba’al whenteal’ c discovers what sheisdoing. as teal’c escapes toanal < unk>, qete sh (156+ kn) each fuel capacity:18@,@ 000 lb( 8@,@ 200 kg) internally, or 26@,@ 000 lb( 12@,@ 000 kg) with two external fuel tanks performance maximum speed: at altitude: mach 2@.@ 25(1@,@500mph,2@,@ 410km/h)[estimated] supercruise: mach 1@.@ 82(1@,@220mph,1@,@ 960km/h)range:> 1@,@ 600 nmi( 1@,@ 840 mi, 2@,@960 Transformer factor 322 in layer 10 with saliency map Explaination: biography, someone born in some year... .onlythreepitchersthrew more com- plete games in majorleaguecareers shorter than getzein’ s nine@-@ year career. getzein had his most extensive playing time withthedetroit wolverines, compiling records of 30@-@ 11 and 29@-@ 13 in 1886 and 1887.in the 1887 world series( which detroit won, 10 games to 5), getzein pitched sixcompletegames andcompileda 4@-@ 2 record with a 2@.@ 48 era.healso won 23 gamesforthebostonbeaneatersin 1890.==early years==getzeinwas bornin1864 andtelegraph lines and networks.the west construction company, based in chattanooga, tennessee, was a general contracting and con- struction firm also involved in the operation and maintenance of railway, telephone, and telegraph lines.== personal life===== marriage andchildren=== on april10,1875,in hampshire county, flournoymarriedfrances" fannie" ann armstrong white(april10,1844 –february25,1922),thedaughterof hampshirecountyclerkofcourtjohnbaker whiteandhis wife frances ann streit white. frances white’ s brother, robert white, served as west virginiaattorney general, and her buffalo, new york businessman who made his fortune infive@-@and@-@ dime stores. hemergedhismore than100stores with those of his first cousins, frank winfield wool- worth and charles woolworth, to formthef. w.woolworth company. he went on to hold prominent positions in the merged company as well as marine trust co. he was thefatherof seymour h. knox i and grandfather of seymour h. knox i and northrup knox, the co@-@ founders of the buffalo sabres in thenationalhockey league.==biography==hewasborn inapril1861inrussell,saintlawrence stars for eighteen years.the american film institute( afi) ranked cooper eleventh on its list of the twenty five greatest male stars of classic hollywoodcinema.== early life== frank jamescooperwas born on may 7,1901,at 730 eleventh avenue in helena, montanato englishimmigrantsalice(neebrazier, 1873 –1967)andcharleshenrycooper(1865 –1946).hisfatheremigratedfrom houghtonregis,bedfordshireandbecame a prominentlawyer, rancher, and eventually a montanasupremecourtjustice.his mother emigrated from gillingham, kent and married charles in montana.in 1906, charles purchased the 600@-@acre orange(1971), which kubrickpulled from circulation in theukfollowing a mass media frenzy — most of his films were nominated for oscars, golden globes, or bafta awards. his last film, eyes wide shut, was completed shortly before his death in 1999.== early life== stanley kubrick was born on july 26, 1928, in thelying@-@ in hospital at 307 second avenue in manhattan, new yorkcity. hewasthe firstoftwochildren ofjacobleonardkubrick(may21,1902 –october 19,1985),knownasjack or jacques,and hiswife sadie gertrude kubrick managed with a catch and release regulation. trophy trout and wild brook trout enhancement regulations apply to the remainder. atotalof 31 classawild trout waters have beendesignated as wilderness trout streams. fishing in class a wild trout waters is permitted year@-@ round, althoughthe killing of fish is forbidden from labor day to the beginningofthe following year’ s trout season.== gallery=== henry bell gilkeson=henrybellgilkeson(june 6,1850–september 29,1921)was anamericanlawyer,politician,school administrator, andbankerin westvirginia. gilkeson wasborn in moorefield, movement, there have been few more remarkable figures thanmarjory stoneman douglas."== early life== marjory stonemanwasbornonapril 7, 189 0,inminneapolis,minnesota,theonly childoffrankbryantstoneman(1857 –1941)andlilliantrefethen(1859– 1912),a concertviolinist.oneofher earliest memories was her father reading to her the song of hiawatha, at which she burst into sobs upon hearing that the tree had to give its life in order to provide hiawatha the wood for a canoe. she was an early and voracious reader amazon.com.=== dvd release====johann mickl=johannmickl(18april1893– 10april1945)wasanaustrian@-@born generalleutnantand division commanderin thegermanarmyduringworldwarii,and was one of only 882recipients of the knight’ s cross of the iron cross with oak leaves.he was commissioned shortly before the outbreak of world war i, and served with austro@-@ hungarian forces on the eastern and italian fronts as company commander in the imperial@-@ royal mountain troops.during world war i he was decorated severaltimes for bravery and leadership,and veryunusual properties, such as a quantum criti- cal point behavior, exotic superconductivity,and high@-@ temperature ferromagnetism.= babe ruth=georgehermanruthjr.(february 6,1895–august 16,1948),better knownas babe ruth,wasanamerican professionalbaseballplayerwhosecareer in majorleague baseball( mlb) spanned 22 seasons, from 1914through1935.nicknamed" the bambino" and" the sultan of swat", he began his mlb career as a stellar left@-@ handed pitcher for the boston red sox,butachieved his greatest fame as a slugging outfielder for the airin regular scheduled services. it includes the city, country, airport and the period in which the airlineserved the airport. hubs aredenoted with a dagger().= william s. taylor= william sylvestertaylor(october10,1853– august2,1928)wasthe33rdgovernor ofkentucky.hewasinitiallydeclared the winner of the disputed gubernatorialelection of1899, but the kentucky general assembly, dominated by thedemocrats, reversed the elec- tion results, giving the victory to his democratic party( united states) opponent,william goebel. taylor served only 50daysas governor. a poorly educated but politically astute lawyer, taylor woodshole, massachusetts, where he stud- ied marine bioluminescence. he also worked at the woods hole oceanographic institution.== early life==georgethomasreynoldswas bornintrenton,newjerseyonmay 27, 1917,thesonof george w.reynolds, a< unk> forthepennsylvaniarailroad,and his wife laura, a secretary with the new jersey department of geology.he attended franklin junior high school in highland park, new jersey, until year 10, and then new brunswick high school.he received a bachelor’ s degree in physics from rutgers university in1939. he then entered princeton university,wherewasawarded == shaughnessywasbornonmarch 6, 1892inst.cloud,minnesota,the secondsonoflucyann(foster)and edwardshaughnessy. he attendednorthst. paul high school, and prior to college, had no athleticexperience. when he attended the univer- sity of minnesota,however, he p layed college football under head coach henry l. williams and alongside halfback bernie bierman.shaughnessy considered williams to befootball’ s greatest teacher, andwilliamsconsidered him to be the best passer fromthemidwest. shaughnessy handled boththepassing and kicking duties for the team. he played on sgregoras likewiseavoidsnegative comments, as do most modern historians.= georgenicol( baseball)= georgeedwardnicol(october 17,1870–august4,1924)wasan americanbaseball pitcher and outfielder who playedthree seasons in major league baseball( mlb).heplayedfor the st.louis browns, chicago colts, pittsburgh pirates and louisville colonels from 1890 to 1894.possessing the rare combination of batting right@-@ handed and throwing left@-@ handed, he served primarily as a right fielder when he did not pitch.signed by thebrowns without having previously played any minor league baseball, nicol made his dispatchedpowell and major benjamin mc- culloch to utah to ease tensions with brigham youngand the mormons. powell assumed his senateseat on his return from utah, just prior to the election of abraham lincoln as president. powell became an outspoken critic of lincoln’s administration, so much so that the kentucky gen- eral assembly asked for his resignation and some of his fellow senators tried to have him expelled from the body.bothgroupslater renounced their actions.powelldied at his homenear henderson,kentuckyshortly following a failed bidto return tothesenatein1867.== earlylife==powellwasbornonoctober 6,1812 nearhenderson, thearmyin 1948.hewas promoted to lieutenant general just before his retirement on 29 february 1948 in recognition of his leadership of the bomb program. by a special act of congress, his date of rank was backdated to 16 july 1945, the date of the trinity nuclear test.groveswent on to become a vice@-@ president at sperry rand.== early life== leslierichardgrovesjr. wasborninalbany, newyork,on17 august 1896,thethirdsonoffourchildrenofa pastor,leslierichardgrovessr.,andhis wife gwenneegriffith.a descendantoffrench huguenots who ,burns died on november 11,1928in brooklyn, new york.== biography== thomasp. burnswasbornonseptember6,1864, inphiladelphia.hisparents,patrickand maryburns, were both irish immigrants.in 1883,burns began his professional baseball career as a pitcher with harrisburg of the minor@-@ league interstate association.onthe year, burns postedan earned run average( era) of 2@.@ 30 over 20 games pitched, 15ofwhich were starts. whenhe wasn’ t pitching, burns played second and third base. burns began the 1884 season playing for thewilmington quicksteps, @ beats".==creditsand personnel== lady gaga – vocals, songwriter and producer redone – songwriter,producer, vocal editing, vocal arrangement, audio engineering, instrumentation, programming, and recording at tour bus in europe trevor muzzy–recording, vocal editing, audio engineering,andaudio mixing at larrabee, north holly wood, los angeles, california gene grimaldi – audio mastering at oasis mastering, burbank, californiacreditsadapted from born thisway album linernotes.==charts===travis jackson=travis calvin jackson(november 2,1903–july27,1987)wasan americanbaseballshortstop. =monsonwasbornonaugust 21, 1927, insalt lakecity,utahtog.spencer monson(1901– 1979)andgladys< unk> monson(1902– 1973).thesecond ofsixchildren,he grew up in a" tight@-@ knit" family — many of his mother’ srelatives living onthesame street and the extended family frequently going on trips together. the family’ s neighborhood included several residents of mexican descent, an environment in which he sayshe developed a love for the mexican people and culture. monso n often spent weekends with relatives on their farms in granger( it.andersonwasa professional accordion player and wrote poetryforvarious american paganmagazines.in 1970, he published his first book of poetry, thorns oftheblood rose, which contained devotional religious poetry dedicated to the goddess; it won the clover international poetry competition award in 1975. anderson continued to promote the feri tradition until his death, at which point aprilniino wasappointedasthenew grandmaster of the tradition.== early life===== childhood:1917–1931=== andersonwas born on may 21,1917atthebuffalo horn ranch inclayton,newmexico.hisparents werehilbartalexanderanderson waselsewhere.he had recently become engaged and bought his first house in hillsborough. franklin and benjamin pierce were among the prominent citizens who welcomed president jackson to the state on his visit in mid@-@ 1833.=== marriage and children===onnovem- ber 19,1834,piercemarriedjanemeans appleton(march12,1806–december 2,1863),thedaughterof jesse appleton, a congregationalministerand former president ofbowdoincollege,andelizabethmeans. the appletons wereprominentwhigs, in contrast with the pierces’ democratic affiliation. jane was shy, devoutly religious, and pro@-@ temperance which took delivery of its eight and last globemaster in november 2015; no. 38 squadron, operating king airs; and the australian army’ s 68 ground liaison section.all units are based at amberley,with the exception of no. 38 squadron, locatedat townsville.= clark shaughnessy= clarkdanielshaughnessy(originallyo’ shaughnessy)(march6,1892–may 15,1970)wasanamericanfootball coachandinnovator.heissometimes called the" father of the t formation" and the original founder of theforwardpass, although that system had previously been used as early as the 1880s Transformer factor 386 in layer 10 with saliency map Explaination: topic: war hewasawardeda companion of theorderof st michael and st georgeforhiscommand ofthe4thmachinegunbattalion,the recommendationofwhichparticularly citing his success during attacksonthe hindenburg line.murray’ s final honour came on 11 july 1919, whenhewas mentioned in despatches for the fourth time, having received his third mention on 31 december 1918. from june to september 1919,murray— along with fellow australian victoria cross recipient william donovan joynt — led parties of aif members onatourof the farming districts of britain and denmark to study agricultural methodsundertheeducation schemes.aftertouring through franceand belgium , fromlarge@-@ calibre shells;one of them, allegedly a 14@-@inch( 356 m) round, blew a large hole inherquarterdeckandwrecked the wardroomandthe gunroom.she also took several hits by light shells that day, and, although she suffered damage tohersuperstructure, her fighting andsteamingcapabilities were not seriously impaired. theshipalsoparticipated in the main attack on the dardanellesforts on 18march.thistimea6@-@ inch( 152m)howitzerbatteryopenedfire onagamemnonandhither 12 timesin25 minutes; five of the . lt. riefkohl,whowas also the firstpuerto rican to graduate from the united statesnaval academy,served as a rearadmiralin world war i. frederick l. riefkohl’ sbrother, rudolph william riefkohl also served.riefkohlwas commissioneda secondlieutenantand assignedtothe63rdheavyartillery regimentinfrancewhere he actively partici- patedinthe meuse@-@ argonneoffensive. accordingtothe united states war department, afterthe war he served as captain of coastal artilleryatthe letterman army medical center in presidio of san francisco, in california( 1918). washingtontimes@-@ herald,which ran the headline" hardy wild@-@ eyed aussies called world’ s finest troops".an article in the chicago dailynewstold itsreadersthat australians" in their realistic attitudetowards power politics, prefertosendtheir boys to fight far overseas rather than fighting a battle in the suburbs of sydney".duringthebattle, wavellhadreceivedacablefromgeneral sirjohndillstressingthepoliticalimpor- tance of such victoriesinthe united states, where presidentfranklin d. rooseveltwasattempting to get the lend@-@ lease act passed. it was finally enacted in march1941 . mackay wrote .he also showed respect for occupied pop- ulationsandnever tolerated pillagingnor violence from his men. asasign a gratitude, he wasofferedgifts several times buthewas often seen refusing and sendingthemback. while oncampaignintyrol, he was recordedto haveaccepted a large sum of money buthe immediately distributed it to the local hospitals. further evidence of his humanity was the ca re that hedisplayed for the lives andwell@-@ being ofhis men, whom he was always reluctant to sacrificefor the sake of glory.overallasa heavycavalrycommander,nansoutywas oneof thebestmenavailableduringthe napoleonic @ 000 troops on 11 february. in march1919, princess matoika and rijndam raced each other from saint@-@nazaire tonewportnews in a friendly competition that received national press coverage in the united states. rijndam, the slower ship,was just able to edge out theprincess— andcut two daysfromher previousfastest crossing time — by appealing to thehonorof thesoldiersofthe133rdfieldartillery( returninghomeaboardthe former holland americaliner)andemploying them as extra stokers forherboilers.onher next trip,the veterantransport loaded troops at saint@-@ nazaire @ july, met his wife in new york,andtogether theytraveled tocolumbus,georgia by way of washington, d.c. and atlanta.==military schools== for the ten years following world war i, troy middleton would be either an instructor or astudent in the successionof militaryschools that army officers attend during their careers. mid- dletonarrivedin columbus,georgiawith strong praisefromhis superiors, and would soon get his efficiency report, inwhichbrigadiergeneral benjaminpooreofthe4thdivisionwrote ofhim,"the best all@-@ around officeri have yet seen.< unk> by his rapid promotion from coaland 700 longtons(710t) of fuel oil andthatprovided her arangeof 3@,@ 500 nautical miles( 6@,@ 500 km) at a speed of 10 knots( 19 km/ h).hermainarmament consistedofa dozen obukhovskii 12@-@ inch(305m)pattern190752@-@ calibregunsmounted in four tripleturrets distributed the lengthofthe ship.therussians didnot believe that super firing turrets offered anyadvantageastheydiscounted the value of axial fire and believed that super firing turrets could not fire while over the lower turret because of ’llstillbe playing from 2007" and awarded it"playstation3@-@ exclusive game of the year".=11th battalion( australia)= the 11th battalion was an australian army battalion that was among the first infantry units raised during world war i forthefirstaustralianimperial force. it was the first battalion recruited in western australia,andfollowing a brieftrainingperiod in perth,the battalion sailed to egypt where it undertook four months of intensive training.in april1915 it took partinthe invasion ofthe gallipolipeninsula,landing at anzac cove.in august1915thebattalionwasinaction inthebattleoflonepine. following wastransferred to westernaustralia,be- ing attached to the 6th brigade, which was based around geraldton.inseptember 1942, as part ofan army@-@wide reduction that came about because of over@-@ mobilisation,thebattalion wasamalgamated with the 14th battalion to become the 14th/ 32nd battalion(prahran/ footscray regiment).inearly1943,the 14th/32nd battalioncarriedoutamphibious warfaretraininginqueenslandbefore being deployed to the buna– gona area in new guinea in july.thebattalionwouldremain in mainland new guineaandnew britain for the next two years, under the command of lieutenant inan allied air raid on 10 december 1941, micklwasappointed totemporarilycommand the division.duringdecember, mickl was woundedinthehead and hand,but remained at his post.rommelrecommendedmickl for the knight’ s cross of the iron cross, forhis leadership at sidi rezegh, and it was duly awarded on 13 december 1941.theharshconditions ofdesertwarfare had begun to affect mickl’ s health, so at the end of december he was sent home on convalescent leave.===eastern front=======12thriflebrigade==== on25 onto bijeljina which was taken against light partisan resistance late on 16 march.the 27thregimentthen consolidated its position in bijeljina whilethe28th regiment andthe divisional reconnaissance battalion( german:< unk>) bore the brunt of the fighting as they advanced through< unk>, celic and koraj at the foot of the majevica mountains.sauberzweig later recordedthatthe2ndbattalionof the28thregiment(i/28)"atcelic stormedthe partisandefenseswith( new) battalion commander hans hanke at the poin t" and thatenemy forceswithdrew after ofmatthews,the company 2ic,who had takencommandalmost immediately after the company commanderwaswounded. under his command,each of the platoons assaulteda different cluster of buildings to whichtheyhad been assigned during trainingonthereplicavil- lage at hastings. the west side boys’ammunition store was found and secured and, once the rest of thebuildingshad beencleared, the paras took updefensivepositionstoblockany potential counter@-@attackand patrols went intothe immediate jungle in search of any west side boyshidingin thebushes.thevillage was completely secure by08:00andtheparas secured the approacheswithclaymore ), increased her metacentricheightto 6@.@ 3 feet( 1@.@ 9 m) at deep load, and all of the changes to her equipment increased hercrew toatotal of 1@,@188.despitethebulges she was able to reach a speedof21@.@ 75 knots(40@.@ 28km/ h; 25@.@ 03 mph).a briefrefitin early 1927sawtheadditionof two morefour@-@inchaagunsandthe removalofthe six@-@ inch gunsfromthe shelterdeck.about 1931,ahigh@-@ angle control became enragedathim, slapping himacross the face. he began yelling:" your nerves, hell, you are just a goddamned coward.shut up that goddamnedcrying. i won’ t havethese brave men who have been shot at seeing this yellow bastard sitting here crying." patton then reportedlyslappedbennett again, knocking his helmet liner off, andorderedthereceiving officer,majorcharlesb.etter,notto admithim.pattonthenthreatenedbennett," you’ re going backtothe front lines and you mayget shot and killed, but you’regoing to fight. ifyoudon ’ t, i’ secondaryguns, two of which were dis- abled.the ammunition storesforthese two guns were set onfireand the magazines had to be flooded to prevent an explosion. the ship neverthelessremainedcombateffective, as her primarybatteryremained in operation,as didmost of her secondary guns; konigcould also steamatclose to her maximum speed. other areas of the ship had to be counter@-@ floodedto maintain stability; 1@,@ 600 tons of water enteredtheship,eitherasaresult ofbattledamageorcounter@-@ flooding efforts.thefloodingrenderedthebattleship sufficiently low in the watertoprevent the ship frombeing able in1924 andriceinstitute, houston,texas in 1928.he dropped out of graduate school after one year and decided to hitchhike to san francisco. thelack of work meant hunger, so hechoseto join the united states army’ s 11th cavalry regiment as a private on july 30, 1930,servingin monterey, california.after a year in the horse cavalry, parrishbecame anaviationcadet in june 1931 and subsequently qua lified as an enlisted pilot. hecompletedflighttrainingin 1932 andwas assigned tothe13thattacksquadronatfort crockett,neargalveston,texas.one yearlater inseptember 1933 parrish during the battle,murray was awarded the victoria cross.soonafter his victoria cross action, he was promoted to major and earned a bar tohisdistinguished service order during an attack on the hindenburg line near bullecourt. promoted to lieutenant colonel in early1918, heassumedcommandofthe4thmachine gunbattalion,wherehe wouldremainuntil theendofthewar.returningto australia in 1920, murray eventuallysettledinqueensland, where he purchased thegrazingfarmthat would be his homeforthe remainder of his life. re@-@enlisting for serviceinthe second world war, he was appo inted as commanding officer 10officers and 315enlistedmen, plus an additional four officers and 19 enlisted men if serving as a flotillaflagship.== construction and career==theshipwasordered on 7 july 1934 and laiddownat deutsche werke, kiel, on 2 january 1935 as yard number< unk>. shewaslaunched on 30november1935 and completed on 8april1937.shewasnamed after maxschultzwhocommandedthe torpedo boat< unk>andwaskilledinactionin january1917.korvettenkapitan martin< unk> was appointed as her first captain. max schultz was assigned to the 1st destroyer division on 26 thecommand of otto von diederichs.the squadron participated in the fall maneuvers in1894, which simulated a two@-@ front war against france andrussia; deutschland’ s squadron acted as the russian fleet during the exercises.between 1894 and 1897, deutsch- land wasrebuiltin the imperial dockyard in wilhelmshaven. theshipwasconvertedinto anarmored cruiser;herheavygunswere removedandreplacedwith lighterweapons, includingeight 15 cm(5@.@ 9 in)and eight 8@.@8cm( 3@.@ 5 in)guns.herentire rigging equipmentwasremoved and two heavy militarymast swere installed called on many times to maintain orderin times of disaster and to keep peace during periods of political unrest.oklahoma governor john c. walton used divisiontroopstoprevent the statelegislaturefrommeetingwhen they were preparing to impeach him in1923. governor william h. murray called out the guard several times during the depression to close banks, distribute foodandonce to force the state of texas to keep openafree bridge over the red river which texas intended to collect tollsfor, even after federalcourtsordered the bridge not beopened.thedivisionwouldgoonto seecombatinworldwariiasoneof four nationalguarddivisions active during Transformer factor 170 in layer 10 with saliency map Explaination: topic: music production 2nd street tunnel and part of downtown los angelesspreadoutover a 48@-@hour period. kesha explained the idea behind thevideo as wellasthe experience duringaninterview withmtvnews; she said that the video was different fromherother videos, notingthatit wasgoingto show a sexier side of herself.the musicvideofor"werwhower"is presentedas an undergroundparty.thevideo startsoff with futuristic flashing lights. kesha, seen in a ponytail wearing gray and black makeup, chains,ripped stockings, and a sparkly one@-@ piece leotard made of shards of broken andseveral european territories), her" end- less love"duetwith luthervandross( number@-@ one in new zealand)and"against allodds"featuringwestlife(number@- @oneintheunitedkingdom)."thankgod i found you" was also omittedfromthe japanese tracklisting, and replaced with" all i want for christmasis you".forthe album artwork, carey launched a social media campaign on april 12,2015, whereby fans had to share a link to her websiteinorder to reveal the cover which was concealed by a curtain.using the hashtag"< unk>", single," we belong together".he contained to add" but still,if mimi ’ s going to mine from her own extensive back catalog ofballads,those arethe primo melodies togofor." a reviewer fordjbooththoughtthat minaj" ruined" the song.=== music video=== theaccompanying musicvideofortheremixof"upout myface"wasdirectedbycarey’ s husband,nickcannon. minaj spoke about filming a video with carey andhowshedidnot believe that the video would ever be released:" i didn ’ t even tell anyonei shot a video with theproducer, few days after hehadfin- ished thecomposition,madonnacompleted writingthelyricsof"idon’tgive a".solveigunderstoodthatthelyrics wereprobablereferencestowards madonna’ s life andthusreceived coverage in the press. however,hewas not aware of the inner meaning behind the lyrics. with billboard magazine,the producer further explained: at first i thought we were going to work on one song; that was the originalplan. let’ s try to work on one song and take it from there– not spend too muchtime thinkingaboutthe l egend, and do something that just makes sense. providedanadditionalandassistant engineering.all the instruments were pro- vided by eriksen and hermansen whiledeansang the background vocals. in may2011, in the mix review, an analyzing commercial productions, mike seniorofsound on soundrevisitedthe original mixingofthesong. according to him, before he startedthemix,seniorplayed the song a couple of times before releasing what thing about it" bugged" him.workingit out, he noted that the harmonyofthemixis underminedby thekickdrum."what’smyname?" containsbasicharmoniesthatarea bar of f minor, a bar of a major practicedintheirbackyards andat< unk>salon, owned byknowles’ s mother, tina.the group would test routinesinthe salon, when it was on montrose boulevard in houston, andsometimes would collect tips from the customers. their try out would be critiqued by the peopleinside. during their school days, girl’ s tymeperformed at local gigs. when summer came, mathew knowles established a" boot camp" to train them in dance andvocallessons. after rigorous training, theybeganperformingas openingactsforestablishedr&bgroups ofthattimesuchas swv, druhilland immature. tina dayreception at the greek embassy.upon returnto greece, she wasgreetedat the airport by fans alongwiththemusicvideoof" mynumberone"playingonthevideo monitors. while ingreece, sheattendedthe openingceremonyofthe european final four for the volleyball champions league in< unk>, whereher song wasplayedasshe appeared on stage with cheerleaders.onmarch29, paparizou arrived in valletta, maltawhereshe signed autographs, appeared on television stations, and gave interviews to the local media. following malta, she traveled to serbia andmontenegro where she gave additional interviews before moving on to and andherlow hip@-@grindduring’ rude boy’ were the smash hits of her body language." deborah linton of city life wrote that rihanna" evenmanagesto make a psychiatric couch looksexy". linton called the show’ s stage sets impressiveand imaginative. rick massimo of the providence journal wrote thatrihanna" looked like a neon@-@ sign rendition of herself during’ rehab’, rarely addressed the audience, and didn’ t rise above flat cliche in that until the very end of theshow"."rehab"andrihanna’s 2009single"russianroulette"were excludedfromthe set onlyafewhours. he said:" there were a lot oftracks, but ijustenjoyed it, to be honest. i knew how i wanted ittosound, and it was pretty much the last song we cut; alotof the mixing was nailed in the production as well,whichhelped. dream did a great job producing this track."the bar oneguitartrackof"schoolin’life" wasentirely programmed. similarly,thelive drumsectionin thehookwas actually done with programmed drums.oncethe mixing was over, swivel’simpression wereasfollows:[’ schoolin’ life] absolutely tourbegan onmarch1, 2000atthe houseof blues in los angeles,while other venues included paris olympia, trump taj mahal, brixton academy, the montreux jazz festival, and the essence jazz festival in new orleans.byjuly, the tour’sfirst half had sold out in each city. the tourlasted nearlyeightmonths,whil e performances wentforup to three hours a night.the voodoo tour was taken internationally, withoneof the most notable performances being the freejazz festivalinbrazil.themusicvideofor" untitled(howdoesitfeel)"portrayed d’ angelo asasex symbol hobsonnoted thatrihanna" rejects the vic- timstance"inthevideofor"man down",andelucidated that she played the role of a rape survivor who shotherattacker. she attributed thelocationof shooting the videoin jamaica as significant, due to how the image of a gun proliferatedduring1990s jamaican dance hall’s to" expressfemalerage".theprologue depictsrihannaasa"dark@-@ hooded" femme fatale whereby the narrative explains her motives for murder and provokes the spectator to sympathizewith her because she danced ina provocativemanner with amanin aclub, which hada deep impactondelonge in that he spent anightupcrying for him when he wrote thetrack."alittle’senough"was inspiredbya religious concept inwhicha god came to bring positive change on earth when it faces terrorism, war or famine."the war",an anthemabout the iraq waranditsdeath toll, is succeeded by"it hurts",atrackabout a friend of delonge with acheatinggirlfriend." it’ saterrible situation where my friendisbeing crushed from the inside out by all the manipulative stuff she’ s doing and this song’ s justtookthatdress out of the storage– it has a 27@-@ foot train and it was just all hand@-@ beaded and stuff and so i figured we might aswellget a use out of it.’=== synop- sis=== the videofeaturescareyreadying for her wedding, and follows her to the altar, as wellas her escape from the reception. many of theactorsfeaturedincarey’ s" it’ s likethat"videowereinthatof"we belongtogether",whichwasshotasa continuationfromthe" it’ s like that"video. it begins with 3in dutch@-@ speaking flanders andnumber2 in french@-@ speaking wallonia.it was certified gold bythebelgianentertainmentassociation( bea) for selling more than 15@,@ 000copies. although the song spent only 1 week on the italian singles chart(at number 8),itwas certified platinum by the federazione industria musicale italiana( fimi) in2014for selling more than 30@,@ 000copies.== musicvideo===== background and synopsis=== anthonymandler directedthemusicvideofor"man down"inapril2011onabeach in atnumbers 18 and43intheunited states, and experienced moderatesuccessworldwide. unlike her previous records, spears didnot heavily promoteblackout;heronly televised appearancefor blackout was auniversally@- @pannedperformanceof"gimme more"atthe2007mtvvideo music awards.== background and development== in november2003, while promoting her fourth studio album in the zone, spears told entertainment weeklythatshe was already writing songs for her next album andwasalso hoping to startherown record label in 2004. henrik jonback confirmed that he hadwrittensongs with her during the european legof the onyx hotel tour," ofalbums also had increased sales due to discounting and publicity generated bythe single and her performance. billboard estimated that her top@-@ 10 digital sales collectively increased over 1@,@ 700 percent.madonna’ s bestsellingalbum wasthe2009 greatest@-@ hitscollection, celebration, which sold 16@,@ 000copies( up 1@,@ 341 percent) and reentered the billboard200album chart. the following weekcelebrationfell 105 spotsonthechartto number 157, with sales falling to4@,@000 copies."givemeallyour luvin’"fell to number39onthe hot openedthe performance with" yeah 3x" andwas dressed in a white formal suit, accom- panied by" full@-@ skirted dancers". brown was eventually joined onstage by tuxedo@-@ clad dancers and began dancing tothe1993 wu@-@ tang clansingle" protect ya neck". his dance routinethenmovedinto1991,wherehe dancedtonirvana’s"smellsliketeen spirit".brown’ s performance then came back to the future,where he began tosing" beautiful people".whileperforming the song, he wassuspendedin the air, and then lowered to another stagewhere hecontinued to registerthat she didn’ t know she had." from the moment shewassigned in the film, madonnahadexpressedinterest inrecording adanceversionof"don’tcryfor meargentina".according to herpublicist liz rosenberg," since she didn’ t write the music and lyrics, she wantedhersignature on that song... i think on her mind, thebestwayto do it was go in the studio andworkup a remix". for this, inaugust 1996,while still mixing the film’ s soundtrack, madonna hired remixers pabloflores and javier garza. according toflores , the singer daccumulated until then but that was in- stead an ideal marriage ofproductionand performance."instead, the red lights on the stageplayed up the" ominous" toneofthe song as it gradually increased its tempo to the point wherebytheend of the song was on the verge of sounding like anincantation.forthe diamondsworldtour,rihannaperformed" mandown"inacaribbean@-@ theme sectionof theshow,whichalso included" you da one"," noloveallowed"," what’ s my name?" and"rude boy".james lachno of the telegraph highlight the caribbean@-@ themed edgeof several realities:thefilm, the dream it inspires, the waking world it illuminates". themusicin" i just can’ t stop loving you", a duetwithsiedah garrett, consisted mainly of finger snaps and timpani."justgoodfriends", a duetwithstevie wonder, was viewed by criticsas sounding good at the beginning of thesong, ending with a" chin@-@ bobbing cheerfulness"."theway youmakeme feel"’ smusic consisted ofbluesharmonies.the lyricsof"anotherpartofme"deal with being united, as"we notmanufactured.noone paid these kids."===liveperformances===one directionperformed"whatmakesyou beautiful"onred or black?on10 september 2011.theperformancestarted with hosts ant&dec announcing that the bandwas supposedly running latefortheir appearance, and cut to a video of onedirectionboardinga london tube carriage full of fans, asthestudio version of the song began playing.each fan on the tube was given a numbered ticket.theband and fans disembarked the tube and made their way to the television studio, wheretheremainder of the song was sunglive . after the song, This is the end of visualization of high-level transformer factor. Click [D] to go back.