You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-04 - 20:36
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Mon May 04, 2026 12:57:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
127119984378,762cyclictest0-21swapper/211:37:382
1271199708241,465cyclictest8284-1kworker/u17:110:31:152
127109970222,127cyclictest0-21swapper/111:12:311
12711996899,677cyclictest0-21swapper/210:46:222
12711996811,677cyclictest0-21swapper/210:25:332
12711996811,677cyclictest0-21swapper/210:25:322
127099966065,380cyclictest0-21swapper/010:51:020
1271599658104,380cyclictest20308-21diskmemload11:04:496
127149965763,211cyclictest698-21ptp4l11:02:345
127109965620,127cyclictest0-21swapper/111:47:021
12709996350,84cyclictest0-21swapper/010:35:000
1271599630118,296cyclictest0-21swapper/611:38:546
127119962833,380cyclictest0-21swapper/211:50:352
127119962833,380cyclictest0-21swapper/211:50:352
127129962777,126cyclictest0-21swapper/312:42:583
127119962125,593cyclictest0-21swapper/211:29:002
127119961966,550cyclictest1026-21ssh11:04:272
127119961922,508cyclictest0-21swapper/212:15:152
127119961922,508cyclictest0-21swapper/212:15:142
1271399612146,126cyclictest12707-21cyclictest11:38:194
127149961065,415cyclictest21716-21nscd10:55:245
127109961018,126cyclictest0-21swapper/112:42:311
127129960855,338cyclictest3192-21ssh10:24:593
127129960855,338cyclictest3192-21ssh10:24:593
127129960855,338cyclictest3192-21ssh10:24:583
127109960816,127cyclictest0-21swapper/111:04:551
127109960815,127cyclictest0-21swapper/112:14:131
127119960715,512cyclictest0-21swapper/212:22:162
127129960310,126cyclictest0-21swapper/311:58:333
12711996005,592cyclictest0-21swapper/208:59:352
127139959743,296cyclictest0-21swapper/409:22:484
127159959684,424cyclictest0-21swapper/608:58:276
12710995961,338cyclictest0-21swapper/112:23:241
1271699594124,254cyclictest0-21swapper/712:43:337
12716995940,84cyclictest0-21swapper/710:38:097
12712995940,84cyclictest0-21swapper/311:46:123
127119959440,383cyclictest0-21swapper/210:38:202
127159959382,338cyclictest0-21swapper/611:56:536
12715995930,84cyclictest0-21swapper/612:44:206
12714995930,84cyclictest0-21swapper/512:13:055
127119959280,381cyclictest14189-21kworker/2:112:44:482
12709995921,2cyclictest0-21swapper/009:23:040
1271399590163,338cyclictest704-21nscd11:50:164
1271399590163,338cyclictest704-21nscd11:50:164
1271299590121,296cyclictest12707-21cyclictest11:14:493
127139958981,126cyclictest0-21swapper/411:18:494
127139958735,337cyclictest49-21ksoftirqd/412:43:074
127109958536,84cyclictest121rcu_preempt11:56:471
127129958228,84cyclictest0-21swapper/311:29:253
127119958169,296cyclictest0-21swapper/212:06:282
127109958173,84cyclictest121rcu_preempt09:25:091
127099957972,126cyclictest0-21swapper/010:47:430
1270999579110,338cyclictest24678-21diskmemload10:37:350
127109957828,126cyclictest0-21swapper/110:51:231
127099957868,338cyclictest0-21swapper/012:12:240
127129957563,296cyclictest0-21swapper/310:53:373
1270999575148,295cyclictest0-21swapper/012:06:590
1271399573104,338cyclictest4958-21ssh10:25:274
1271399573104,338cyclictest4958-21ssh10:25:264
127119957219,169cyclictest17394-21ssh11:13:132
1271399571147,292cyclictest0-21swapper/408:59:074
127129956919,296cyclictest0-21swapper/312:11:103
1270999569101,295cyclictest0-21swapper/011:29:020
127139956818,126cyclictest0-21swapper/411:47:384
1271299568100,295cyclictest0-21swapper/311:05:043
127119956815,550cyclictest0-21swapper/209:22:322
127149956759,126cyclictest0-21swapper/510:38:525
1271499567225,169cyclictest12561-21rm11:50:225
1271499567225,169cyclictest12561-21rm11:50:225
127149956717,84cyclictest121rcu_preempt10:52:325
127159956249,296cyclictest0-21swapper/611:48:046
127139956253,506cyclictest0-21swapper/409:00:204
127139956253,506cyclictest0-21swapper/409:00:194
127109956213,126cyclictest12707-21cyclictest10:39:171
1270999561477,82cyclictest101ktimersoftd/012:21:150
127169956092,465cyclictest0-21swapper/710:54:477
127129956093,84cyclictest121rcu_preempt10:33:103
127119955948,508cyclictest0-21swapper/210:54:472
12713995573,127cyclictest20766-21/usr/sbin/munin10:35:204
127169955648,126cyclictest0-21swapper/711:48:037
1271399555214,338cyclictest722-21nscd10:23:234
1271399555214,338cyclictest722-21nscd10:23:224
1271399555214,338cyclictest722-21nscd10:23:224
12713995551,338cyclictest0-21swapper/410:49:444
127149955284,168cyclictest9004-21ssh11:47:225
127099955283,295cyclictest0-21swapper/011:15:320
12715995510,84cyclictest0-21swapper/610:22:456
12715995510,84cyclictest0-21swapper/610:22:446
12715995510,84cyclictest0-21swapper/610:22:446
12713995511,377cyclictest0-21swapper/409:25:154
127169955085,126cyclictest0-21swapper/711:14:247
127169955038,296cyclictest0-21swapper/711:03:217
12710995501,302cyclictest0-21swapper/109:24:441
127139954980,338cyclictest0-21swapper/412:21:384
127159954836,296cyclictest0-21swapper/610:47:216
127159954680,176cyclictest0-21swapper/610:33:576
127139954680,85cyclictest121rcu_preempt12:11:284
127109954534,296cyclictest0-21swapper/111:17:161
127099954538,295cyclictest0-21swapper/011:47:500
1271599544289,127cyclictest21123-21ssh12:13:456
127119954434,507cyclictest0-21swapper/211:46:242
127149954345,75cyclictest0-21swapper/511:56:435
1271299541119,126cyclictest41-21ksoftirqd/309:23:253
1271199541287,83cyclictest321ktimersoftd/212:11:552
1271099541116,295cyclictest0-21swapper/110:47:041
127129954033,84cyclictest0-21swapper/312:25:003
127119953979,334cyclictest20872-21grep11:15:272
127109953931,126cyclictest0-21swapper/111:40:201
127149953828,337cyclictest121rcu_preempt10:47:235
1270999537116,253cyclictest0-21swapper/011:04:170
127159953669,127cyclictest0-21swapper/612:06:066
127159953568,85cyclictest0-21swapper/611:15:586
1271399535236,85cyclictest0-21swapper/411:11:464
127129953569,211cyclictest0-21swapper/308:57:513
127159953425,84cyclictest0-21swapper/610:25:076
127159953425,84cyclictest0-21swapper/610:25:066
127099953223,338cyclictest0-21swapper/011:40:150
127149953019,88cyclictest121rcu_preempt12:42:235
1271499529231,126cyclictest0-21swapper/509:00:195
1271499529231,126cyclictest0-21swapper/509:00:185
127109952822,83cyclictest0-21swapper/111:38:491
1270999528144,169cyclictest0-21swapper/012:43:200
127139952761,127cyclictest0-21swapper/411:28:454
1271099527103,126cyclictest18963-1kworker/u17:211:28:301
127099952761,211cyclictest0-21swapper/007:35:180
1271499526185,126cyclictest0-21swapper/511:27:425
127169952413,508cyclictest0-21swapper/712:23:377
127149952413,338cyclictest0-21swapper/510:32:535
127149952357,211cyclictest0-21swapper/512:00:225
1271499521182,126cyclictest121rcu_preempt12:22:145
127099952114,83cyclictest121rcu_preempt11:57:420
127149952094,126cyclictest17392-21ssh11:13:135
127159951953,211cyclictest0-21swapper/610:52:486
12714995179,84cyclictest0-21swapper/512:06:425
12712995165,126cyclictest0-21swapper/312:07:103
127129951549,84cyclictest0-21swapper/311:40:143
12713995133,507cyclictest0-21swapper/410:54:464
12711995121,126cyclictest0-21swapper/211:56:222
1271599511170,126cyclictest121rcu_preempt11:29:186
12712995114,84cyclictest0-21swapper/311:30:183
12711995111,42cyclictest0-21swapper/210:22:252
12711995111,42cyclictest0-21swapper/210:22:242
12711995111,42cyclictest0-21swapper/210:22:242
127099951183,84cyclictest0-21swapper/011:30:240
127169951038,296cyclictest0-21swapper/710:33:037
127109951043,464cyclictest0-21swapper/110:31:571
12712995090,84cyclictest0-21swapper/310:49:373
1271099509125,126cyclictest27518-21turbostat10:24:551
1271099509125,126cyclictest27518-21turbostat10:24:551
1271099509125,126cyclictest27518-21turbostat10:24:551
127159950840,295cyclictest0-21swapper/609:22:506
12714995081,295cyclictest0-21swapper/511:15:295
1271699507124,126cyclictest73-21ksoftirqd/712:13:587
127129950780,211cyclictest0-21swapper/311:19:483
127169950680,296cyclictest121rcu_preempt10:46:267
127159950679,211cyclictest65-21ksoftirqd/612:24:136
127109950640,126cyclictest121rcu_preempt09:00:191
127109950640,126cyclictest121rcu_preempt09:00:181
127099950641,126cyclictest0-21swapper/011:37:360
1271599505121,295cyclictest0-21swapper/610:38:186
127139950580,210cyclictest121rcu_preempt12:06:574
127169950476,212cyclictest0-21swapper/711:50:317
127169950476,212cyclictest0-21swapper/711:50:307
127149950434,43cyclictest0-21swapper/509:24:045
12709995041,245cyclictest0-21swapper/010:24:520
12709995041,245cyclictest0-21swapper/010:24:510
12709995041,245cyclictest0-21swapper/010:24:510
127139950280,338cyclictest0-21swapper/407:33:594
127099950236,42cyclictest121rcu_preempt08:59:060
127109950133,295cyclictest0-21swapper/112:15:051
127109950133,295cyclictest0-21swapper/112:15:051
127169950073,211cyclictest22109-21ssh11:16:327
127169950073,211cyclictest0-21swapper/709:25:347
1271699500205,210cyclictest20308-21diskmemload11:56:217
127139950036,259cyclictest0-21swapper/411:20:164
127129950034,84cyclictest0-21swapper/310:35:453
127169949935,380cyclictest0-21swapper/711:29:007
127129949973,423cyclictest0-21swapper/311:38:123
127109949933,126cyclictest0-21swapper/110:25:311
127109949933,126cyclictest0-21swapper/110:25:311
127149949731,84cyclictest0-21swapper/509:25:055
127139949732,296cyclictest0-21swapper/411:04:504
127099949630,84cyclictest0-21swapper/012:15:050
127099949630,84cyclictest0-21swapper/012:15:040
127149949328,127cyclictest0-21swapper/507:34:245
127099949329,84cyclictest0-21swapper/011:14:240
127119948965,381cyclictest16890-21ssh09:25:042
127169948819,296cyclictest0-21swapper/710:25:237
127169948819,296cyclictest0-21swapper/710:25:227
1271499487107,126cyclictest3216-21turbostat10:25:025
1271499487107,126cyclictest3216-21turbostat10:25:015
1271499487107,126cyclictest3216-21turbostat10:25:015
127149948622,42cyclictest0-21swapper/511:36:125
1271399486146,338cyclictest64450irq/122-eth011:55:554
127129948621,296cyclictest0-21swapper/312:15:073
127129948621,296cyclictest0-21swapper/312:15:063
1271299486106,42cyclictest0-21swapper/310:55:123
1271199486144,126cyclictest121rcu_preempt11:40:212
127169948516,296cyclictest0-21swapper/711:39:127
127109948516,338cyclictest0-21swapper/112:07:111
127109948516,338cyclictest0-21swapper/108:59:061
127139948420,2cyclictest20292-21munin-run10:35:044
127119948217,386cyclictest0-21swapper/209:50:152
127099948216,85cyclictest0-21swapper/011:50:440
127099948216,85cyclictest0-21swapper/011:50:430
127169947856,338cyclictest0-21swapper/707:33:587
127099947895,380cyclictest15792-21cut09:00:200
127099947895,380cyclictest15792-21cut09:00:200
127129947750,254cyclictest0-21swapper/310:25:263
127129947750,254cyclictest0-21swapper/310:25:253
12711994756,42cyclictest0-21swapper/211:20:292
127139947410,211cyclictest0-21swapper/407:35:014
127129947247,84cyclictest0-21swapper/309:00:153
127129947247,84cyclictest0-21swapper/309:00:143
127099947247,168cyclictest0-21swapper/011:20:290
12712994711,89cyclictest0-21swapper/309:50:073
1271299470213,84cyclictest0-21swapper/309:31:473
127159946942,296cyclictest0-21swapper/612:00:236
12711994691,41cyclictest0-21swapper/207:35:182
1270999468168,126cyclictest121rcu_preempt11:05:100
1271299467252,126cyclictest18682-21kworker/u16:111:20:103
127109946741,253cyclictest0-21swapper/112:00:221
12710994661,211cyclictest0-21swapper/110:55:231
127099946650,79cyclictest28276-21runrttasks07:34:090
1271699465124,338cyclictest16361-21ssh09:24:517
12714994650,82cyclictest121rcu_preempt10:25:255
12714994650,82cyclictest121rcu_preempt10:25:245
127169946239,296cyclictest0-21swapper/711:20:277
1271199461166,42cyclictest121rcu_preempt12:00:162
127109946177,254cyclictest0-21swapper/111:50:441
127109946177,254cyclictest0-21swapper/111:50:431
1271199459120,84cyclictest0-21swapper/207:32:452
1270999459160,92cyclictest18593-21ssh09:25:230
127169945834,84cyclictest0-21swapper/710:22:407
127169945834,84cyclictest0-21swapper/710:22:397
127169945834,84cyclictest0-21swapper/710:22:397
1271699458159,84cyclictest121rcu_preempt12:05:587
1271499458119,337cyclictest0-21swapper/511:20:105
1271199457116,338cyclictest0-21swapper/209:55:152
127109945528,337cyclictest0-21swapper/111:20:271
127169945469,211cyclictest14253-21taskset08:59:087
127159945470,168cyclictest65-21ksoftirqd/611:12:236
127139945475,168cyclictest0-21swapper/411:30:204
1271399453111,339cyclictest0-21swapper/412:15:134
1271399453111,339cyclictest0-21swapper/412:15:134
1271099453113,338cyclictest0-21swapper/109:50:171
127099945327,211cyclictest0-21swapper/010:55:100
1271099452112,338cyclictest0-21swapper/110:40:181
1271099451111,338cyclictest0-21swapper/109:31:461
127149944965,211cyclictest0-21swapper/508:58:235
127119944923,423cyclictest0-21swapper/210:55:192
1271099449109,338cyclictest0-21swapper/112:25:111
127159944863,211cyclictest0-21swapper/607:33:226
1271199448109,84cyclictest0-21swapper/211:05:102
1271199447108,337cyclictest0-21swapper/210:40:162
127169944623,339cyclictest0-21swapper/710:40:117
127169944617,211cyclictest73-21ksoftirqd/709:55:137
1271499446107,337cyclictest0-21swapper/507:35:275
1271699443103,338cyclictest0-21swapper/712:25:107
1271099441101,338cyclictest12707-21cyclictest07:34:241
127099944158,210cyclictest4994-21/usr/sbin/munin10:25:280
127099944158,210cyclictest4994-21/usr/sbin/munin10:25:270
127099943917,219cyclictest3344-21diskmemload09:50:240
127099943695,338cyclictest0-21swapper/012:00:220
127159943593,253cyclictest0-21swapper/609:00:196
127159943593,253cyclictest0-21swapper/609:00:186
127139943596,84cyclictest10209-21/usr/sbin/munin09:55:114
127119943595,338cyclictest0-21swapper/212:25:112
127109943594,126cyclictest0-21swapper/111:30:231
1271699434179,253cyclictest721ktimersoftd/712:00:227
127099943493,126cyclictest0-21swapper/010:40:080
127119943151,42cyclictest0-21swapper/209:31:432
127169943088,339cyclictest0-21swapper/711:40:237
1271199429130,126cyclictest33-21ksoftirqd/209:00:192
1271199429130,126cyclictest33-21ksoftirqd/209:00:182
12714994271,46cyclictest0-21swapper/510:40:085
12712994271,338cyclictest41-21ksoftirqd/307:34:373
127109942795,161cyclictest0-21swapper/107:35:021
127169942584,338cyclictest12707-21cyclictest07:35:117
127139942542,211cyclictest30403-21/usr/sbin/munin12:00:174
127129942583,211cyclictest121rcu_preempt12:25:123
127169942482,339cyclictest0-21swapper/709:00:197
127169942482,339cyclictest0-21swapper/709:00:187
127169942382,339cyclictest0-21swapper/710:55:217
127159942385,84cyclictest31737-21ssh09:31:476
127159942383,338cyclictest0-21swapper/611:20:196
127149942382,338cyclictest0-21swapper/509:55:115
127109942339,84cyclictest121rcu_preempt09:55:171
127169942282,338cyclictest2751-21hddtemp_smartct09:50:167
127169942181,338cyclictest12707-21cyclictest09:31:487
127159942183,83cyclictest0-21swapper/609:25:136
127159942182,84cyclictest0-21swapper/612:25:066
127159942182,84cyclictest0-21swapper/610:55:256
1271599421249,48cyclictest0-21swapper/611:50:346
1271599421249,48cyclictest0-21swapper/611:50:336
127149942180,338cyclictest0-21swapper/512:15:025
127149942180,338cyclictest0-21swapper/512:15:025
127119942180,338cyclictest0-21swapper/211:30:202
127169942083,253cyclictest0-21swapper/711:30:217
127159942082,83cyclictest0-21swapper/609:50:226
127159942081,84cyclictest0-21swapper/609:55:136
127139942041,295cyclictest0-21swapper/409:50:264
127149941939,296cyclictest0-21swapper/512:25:065
127139941980,337cyclictest0-21swapper/411:40:144
127139941977,339cyclictest0-21swapper/410:40:074
127129941978,338cyclictest0-21swapper/311:50:183
127129941978,338cyclictest0-21swapper/311:50:173
127159941878,84cyclictest0-21swapper/607:35:096
127129941877,338cyclictest10325-21cpuspeed_turbos09:55:133
127099941779,335cyclictest0-21swapper/009:31:460
127139941675,338cyclictest0-21swapper/412:25:074
127139941536,295cyclictest0-21swapper/409:31:434
127129941477,84cyclictest28155-21idleruntime10:40:173
127099941434,84cyclictest0-21swapper/009:55:140
1271499413114,126cyclictest27770-21chrt11:40:155
127129941372,338cyclictest0-21swapper/307:35:133
127139941272,338cyclictest0-21swapper/411:05:084
127129941230,126cyclictest0-21swapper/312:00:183
127159941172,84cyclictest0-21swapper/610:40:186
127139941069,338cyclictest0-21swapper/410:55:094
127129940970,84cyclictest0-21swapper/309:25:063
127169940767,338cyclictest0-21swapper/712:15:027
127169940767,338cyclictest0-21swapper/712:15:027
127159940769,84cyclictest0-21swapper/611:40:146
127149940766,126cyclictest0-21swapper/509:50:225
127159940667,84cyclictest0-21swapper/611:30:186
127109940666,338cyclictest18682-21kworker/u16:111:05:101
127099940522,211cyclictest0-21swapper/012:25:120
127169940260,339cyclictest0-21swapper/711:05:147
127149940226,373cyclictest11311-21ssh11:30:205
127159939960,84cyclictest0-21swapper/612:15:146
127159939960,84cyclictest0-21swapper/612:15:136
127129939859,84cyclictest0-21swapper/311:05:083
1271599393136,84cyclictest121rcu_preempt11:05:096
127149938443,338cyclictest0-21swapper/511:05:085
1271499373116,254cyclictest0-21swapper/509:31:465
10228211289,3sleep00-21swapper/007:05:150
1102921110,2sleep40-21swapper/407:05:174
11174210096,2sleep50-21swapper/507:05:195
1221729668,5sleep70-21swapper/707:09:077
1202528845,39sleep20-21swapper/207:06:212
1213928353,27sleep30-21swapper/307:07:573
1220728252,27sleep60-21swapper/607:08:586
1204428049,27sleep10-21swapper/107:06:361
273362710,0sleep40-21swapper/409:12:564
276832690,0sleep00-21swapper/009:46:150
168342690,0sleep10-21swapper/110:14:381
199492680,0sleep00-21swapper/008:15:000
240912620,1sleep0111rcuc/007:25:170
12716995121,26cyclictest67682sleep709:36:047
12716993327,5cyclictest0-21swapper/707:59:237
12716993310,22cyclictest0-21swapper/707:49:357
12709993322,7cyclictest3661-21sort09:35:140
12716993228,3cyclictest0-21swapper/707:22:007
12716993226,6cyclictest0-21swapper/708:44:297
12716993210,21cyclictest0-21swapper/710:19:307
1271599326,15cyclictest65-21ksoftirqd/609:35:326
12716993125,5cyclictest0-21swapper/708:20:497
12711993122,5cyclictest4629-21grep09:35:182
12716993026,4cyclictest0-21swapper/709:19:567
12714993023,5cyclictest3487-21sh09:35:105
1271299309,13cyclictest41-21ksoftirqd/309:35:173
12710993024,3cyclictest3406-21rm09:35:061
12716992910,19cyclictest0-21swapper/707:13:557
12713992924,3cyclictest6590-21ssh09:35:544
12710992910,18cyclictest0-21swapper/107:14:061
12712992810,18cyclictest0-21swapper/308:43:593
12712992810,18cyclictest0-21swapper/308:33:203
12712992810,18cyclictest0-21swapper/307:13:493
12712992810,0cyclictest0-21swapper/310:19:243
12712992810,0cyclictest0-21swapper/307:49:273
12712992810,0cyclictest0-21swapper/307:22:173
12711992810,18cyclictest0-21swapper/207:49:302
12711992810,18cyclictest0-21swapper/207:13:322
12716992727,0cyclictest0-21swapper/708:08:257
12716992710,17cyclictest0-21swapper/708:32:547
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional