You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-10 - 10:31
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Tue Mar 10, 2026 00:53:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31932997297,84cyclictest121rcu_preempt23:25:480
319379966471,126cyclictest0-21swapper/522:53:185
319389963639,380cyclictest0-21swapper/622:33:046
3193799636590,43cyclictest551rcuc/523:25:495
319379963178,343cyclictest0-21swapper/500:43:135
319359962369,381cyclictest0-21swapper/323:54:113
3193599621113,505cyclictest19117-1kworker/u17:100:42:133
319379961968,85cyclictest121rcu_preempt22:31:025
3193899615106,506cyclictest0-21swapper/623:33:366
319379961161,126cyclictest0-21swapper/522:08:325
319339960855,380cyclictest0-21swapper/122:54:361
319339960597,126cyclictest0-21swapper/123:54:331
319359960493,508cyclictest0-21swapper/323:33:363
31938995940,84cyclictest0-21swapper/600:03:406
319339958775,84cyclictest121rcu_preempt21:14:481
319389958430,126cyclictest121rcu_preempt00:30:256
319379958170,338cyclictest0-21swapper/523:50:595
319329958127,338cyclictest0-21swapper/000:28:100
319339957064,168cyclictest25-21ksoftirqd/121:15:171
319349956516,84cyclictest0-21swapper/223:28:562
319329956597,296cyclictest0-21swapper/023:47:530
319339956213,338cyclictest0-21swapper/122:32:241
31936995618,337cyclictest0-21swapper/422:07:164
319389955788,296cyclictest0-21swapper/623:27:066
31936995575,126cyclictest22193-21ssh23:52:284
31938995541,84cyclictest0-21swapper/600:05:166
319399955082,465cyclictest0-21swapper/700:28:277
319389955082,253cyclictest65-21ksoftirqd/600:43:056
319399954980,465cyclictest0-21swapper/700:41:417
319399954880,465cyclictest0-21swapper/723:34:417
319389954837,338cyclictest0-21swapper/621:28:236
319389954779,465cyclictest0-21swapper/622:11:406
319369954778,211cyclictest49-21ksoftirqd/422:15:214
319329954738,127cyclictest0-21swapper/023:33:350
3193399544118,253cyclictest0-21swapper/123:26:241
319349954233,127cyclictest131rcu_sched21:49:542
3193999540113,169cyclictest121rcu_preempt22:12:507
319349954027,254cyclictest0-21swapper/223:51:092
319349953869,338cyclictest32522-21snmpd22:30:242
319339953869,381cyclictest0-21swapper/123:48:581
319339953535,77cyclictest0-21swapper/122:08:251
319349953466,465cyclictest0-21swapper/222:08:442
319379953364,296cyclictest0-21swapper/523:39:465
319389953226,49cyclictest0-21swapper/622:08:586
319369953225,338cyclictest0-21swapper/423:26:034
319359953223,126cyclictest0-21swapper/323:40:283
319379953022,126cyclictest0-21swapper/523:33:365
319349952921,169cyclictest33-21ksoftirqd/223:37:462
3193899527186,338cyclictest6238-1kworker/u17:022:52:296
3193899527101,211cyclictest65-21ksoftirqd/621:14:486
319359952555,296cyclictest121rcu_preempt00:20:123
319349952514,338cyclictest0-21swapper/221:13:062
319359952457,127cyclictest0-21swapper/322:05:533
319399952395,211cyclictest73-21ksoftirqd/700:05:337
319329952215,211cyclictest0-21swapper/021:27:290
31934995168,127cyclictest0-21swapper/223:32:432
31939995132,339cyclictest0-21swapper/723:54:317
31936995093,86cyclictest0-21swapper/422:30:194
31936995090,84cyclictest0-21swapper/423:33:524
31935995090,84cyclictest0-21swapper/300:03:403
31937995080,84cyclictest10260-21ssh00:03:405
3193399507123,211cyclictest25-21ksoftirqd/123:39:231
3193799506165,338cyclictest31930-21cyclictest00:05:025
3193899505164,338cyclictest31930-21cyclictest23:54:336
319329950540,84cyclictest0-21swapper/022:13:160
3193999504119,211cyclictest16886-21ssh21:14:507
319339950482,126cyclictest0-21swapper/123:40:281
3193999503119,126cyclictest73-21ksoftirqd/722:31:537
319379950240,247cyclictest0-21swapper/522:14:345
319369950279,337cyclictest0-21swapper/421:14:484
3193999501245,254cyclictest721ktimersoftd/722:08:447
3193699501246,252cyclictest68350irq/122-eth021:49:574
3193399501122,295cyclictest0-21swapper/119:59:011
3193299498111,259cyclictest0-21swapper/023:35:520
3193399497118,295cyclictest7551-21diskmemload00:42:231
319369949673,169cyclictest0-21swapper/422:13:324
319339949670,84cyclictest121rcu_preempt00:29:381
319389949471,253cyclictest0-21swapper/623:10:056
319359949369,84cyclictest31930-21cyclictest21:28:113
3193399493108,126cyclictest25-21ksoftirqd/123:34:311
319329949371,84cyclictest0-21swapper/022:55:370
3193699492111,126cyclictest0-21swapper/420:59:044
319329949223,380cyclictest0-21swapper/022:34:270
319349949169,256cyclictest7551-21diskmemload23:40:042
319349949122,296cyclictest0-21swapper/222:13:082
3193399491107,171cyclictest0-21swapper/121:49:561
3193899490107,380cyclictest21456-21ssh21:50:016
319379949067,126cyclictest0-21swapper/500:29:315
3193799490110,126cyclictest13226-21ssh22:55:075
319339949025,42cyclictest0-21swapper/122:12:481
319349948962,169cyclictest30696-21python00:30:262
3193499489109,84cyclictest0-21swapper/222:55:212
319369948865,169cyclictest0-21swapper/400:41:194
319369948863,422cyclictest0-21swapper/400:05:164
319329948560,126cyclictest0-21swapper/021:13:540
3193599484227,254cyclictest0-21swapper/323:27:463
319349948416,84cyclictest0-21swapper/223:48:382
319379948117,50cyclictest17163-21sh23:50:005
319379948011,296cyclictest0-21swapper/500:30:155
319369948012,465cyclictest0-21swapper/423:35:534
319329948058,84cyclictest0-21swapper/023:50:040
3193299479180,126cyclictest9-21ksoftirqd/022:06:300
31936994789,254cyclictest0-21swapper/400:04:304
319369947850,253cyclictest0-21swapper/423:49:164
319369947755,131cyclictest121rcu_preempt22:54:484
3193699477139,84cyclictest68350irq/122-eth000:19:474
319379947494,84cyclictest0-21swapper/523:10:325
319349947497,375cyclictest31930-21cyclictest19:59:532
319329947453,84cyclictest0-21swapper/000:42:080
319329947452,84cyclictest0-21swapper/022:52:180
3193799473132,338cyclictest20712-1kworker/u17:021:27:495
319359947250,84cyclictest0-21swapper/300:30:093
31933994723,211cyclictest0-21swapper/121:50:041
319329947288,126cyclictest9-21ksoftirqd/000:05:360
31934994711,84cyclictest0-21swapper/200:04:292
319329946843,210cyclictest9-21ksoftirqd/019:59:160
31933994671,127cyclictest0-21swapper/120:00:171
31933994671,126cyclictest0-21swapper/100:19:481
3193299467130,126cyclictest0-21swapper/020:00:160
31933994661,48cyclictest0-21swapper/122:15:121
319399946581,211cyclictest73-21ksoftirqd/722:55:217
3193899465124,338cyclictest623-21nscd21:15:206
319399946482,126cyclictest0-21swapper/720:59:047
319399946480,211cyclictest73-21ksoftirqd/723:38:287
3193899464123,338cyclictest1931-21ssh22:15:086
3193799464123,338cyclictest7551-21diskmemload00:37:565
3193499464122,169cyclictest0-21swapper/200:28:202
3193899463122,338cyclictest0-21swapper/623:49:586
3193899463121,339cyclictest0-21swapper/600:28:036
319369946340,257cyclictest25921-21ssh00:27:574
3193599463122,338cyclictest18897-21sh21:15:203
3193399463122,338cyclictest0-21swapper/120:59:551
319359946239,169cyclictest41-21ksoftirqd/323:49:423
3193499462121,338cyclictest0-21swapper/221:29:422
3193399462121,338cyclictest0-21swapper/100:30:111
3193299462121,338cyclictest0-21swapper/021:48:360
319399946138,257cyclictest0-21swapper/720:00:157
3193899461119,340cyclictest0-21swapper/623:37:366
319379946037,256cyclictest0-21swapper/523:40:045
319359946076,84cyclictest0-21swapper/323:37:353
319349946033,211cyclictest18686-21/usr/sbin/munin21:15:182
319389945974,211cyclictest65-21ksoftirqd/620:58:046
319369945934,422cyclictest0-21swapper/422:55:274
319349945937,256cyclictest0-21swapper/222:53:062
3193799458118,338cyclictest597-21irqbalance21:14:535
3193799458116,339cyclictest0-21swapper/521:48:145
3193499458117,338cyclictest0-21swapper/222:15:242
3193399458117,338cyclictest0-21swapper/100:05:171
319389945736,83cyclictest0-21swapper/623:55:036
319359945777,84cyclictest0-21swapper/319:59:013
319349945773,381cyclictest0-21swapper/200:42:132
319399945633,256cyclictest0-21swapper/722:52:147
319359945672,211cyclictest41-21ksoftirqd/321:14:403
319359945534,85cyclictest0-21swapper/322:52:463
319399945431,256cyclictest0-21swapper/719:59:017
319389945432,3cyclictest0-21swapper/619:58:146
3193699454114,338cyclictest1556-21runrttasks21:15:234
319329945470,212cyclictest0-21swapper/000:30:100
3193399453112,338cyclictest0-21swapper/122:55:331
3193399453112,338cyclictest0-21swapper/120:29:331
3193999452112,338cyclictest16250irq/123-ahci[0021:48:147
319369945233,124cyclictest0-21swapper/421:28:494
3193399452112,337cyclictest0-21swapper/121:27:491
319399945168,380cyclictest0-21swapper/723:26:027
319359945170,84cyclictest0-21swapper/322:31:033
319359945066,211cyclictest0-21swapper/322:11:013
3193399450109,338cyclictest0-21swapper/122:29:541
319399944926,256cyclictest0-21swapper/723:40:027
319389944827,83cyclictest0-21swapper/623:09:586
3193799448106,339cyclictest0-21swapper/520:29:335
319369944869,295cyclictest0-21swapper/423:55:034
319349944867,296cyclictest0-21swapper/223:55:032
319349944623,256cyclictest0-21swapper/223:09:282
319329944621,211cyclictest9-21ksoftirqd/022:15:150
319329944522,177cyclictest0-21swapper/021:15:200
319339944320,3cyclictest0-21swapper/123:09:281
319399944262,296cyclictest0-21swapper/723:55:027
319369944258,211cyclictest0-21swapper/423:10:314
319349944220,3cyclictest0-21swapper/220:00:142
319329944220,3cyclictest30807-21diskmemload23:40:040
3193599439144,211cyclictest9585-21timerandwakeup22:35:233
319379943795,84cyclictest0-21swapper/521:50:065
319369943715,259cyclictest0-21swapper/423:40:044
319379943614,3cyclictest0-21swapper/523:09:285
3193499436137,296cyclictest31930-21cyclictest20:59:032
319379943449,296cyclictest0-21swapper/522:15:245
319339943411,8cyclictest0-21swapper/123:10:071
319329943495,84cyclictest0-21swapper/000:04:240
319389943292,126cyclictest15569-21cat22:55:316
319379943251,84cyclictest0-21swapper/520:57:375
319329943210,169cyclictest9-21ksoftirqd/021:50:050
31939994292,211cyclictest10414-21ssh00:03:467
31935994297,2cyclictest0-21swapper/321:49:513
319339942848,84cyclictest0-21swapper/100:03:471
319369942687,85cyclictest0-21swapper/400:30:264
319399942583,339cyclictest0-21swapper/723:48:577
31939994250,84cyclictest0-21swapper/721:15:127
31938994253,256cyclictest0-21swapper/623:40:036
319369942483,338cyclictest0-21swapper/423:09:264
31935994241,84cyclictest0-21swapper/322:15:253
31933994248,296cyclictest0-21swapper/121:00:051
319399942382,338cyclictest0-21swapper/722:35:257
319399942382,338cyclictest0-21swapper/722:15:247
319389942382,338cyclictest0-21swapper/620:29:336
319389942382,338cyclictest0-21swapper/600:20:136
319359942382,253cyclictest0-21swapper/322:55:213
319389942281,126cyclictest31275-21ssh22:29:536
319359942283,84cyclictest0-21swapper/300:28:263
3193499422127,83cyclictest33-21ksoftirqd/223:10:072
319329942283,84cyclictest0-21swapper/020:29:340
319399942180,338cyclictest0-21swapper/723:10:057
319399942180,338cyclictest0-21swapper/700:37:557
3193699421123,126cyclictest49-21ksoftirqd/422:35:234
319349942181,338cyclictest0-21swapper/222:30:002
319359942080,84cyclictest0-21swapper/323:10:073
319349942080,338cyclictest0-21swapper/200:37:552
319329942081,84cyclictest0-21swapper/023:10:070
319399941940,295cyclictest0-21swapper/700:30:167
3193899419246,84cyclictest0-21swapper/622:35:266
319369941978,338cyclictest0-21swapper/420:29:344
319369941978,126cyclictest0-21swapper/419:59:154
319349941980,337cyclictest0-21swapper/200:05:282
319349941977,339cyclictest0-21swapper/221:00:102
319329941981,84cyclictest0-21swapper/021:00:100
319329941981,83cyclictest0-21swapper/023:55:040
319339941879,84cyclictest11084-21wc00:20:191
319389941776,338cyclictest0-21swapper/621:50:066
319379941779,84cyclictest0-21swapper/500:20:175
319329941636,84cyclictest0-21swapper/023:09:580
319399941578,256cyclictest0-21swapper/700:20:087
319399941574,338cyclictest0-21swapper/721:27:547
319359941575,84cyclictest0-21swapper/320:58:563
319399941474,338cyclictest0-21swapper/721:00:037
319369941431,213cyclictest49-21ksoftirqd/400:20:184
319339941475,84cyclictest0-21swapper/122:35:261
319399941373,338cyclictest0-21swapper/721:50:117
319389941131,3cyclictest0-21swapper/620:00:156
319379941174,85cyclictest0-21swapper/521:15:115
319379941171,84cyclictest0-21swapper/521:00:105
3193799411116,211cyclictest0-21swapper/519:59:015
319369941131,171cyclictest0-21swapper/420:00:024
319359941171,84cyclictest0-21swapper/321:50:083
319349941170,211cyclictest0-21swapper/220:29:342
319399941069,338cyclictest0-21swapper/700:19:527
319369941029,297cyclictest0-21swapper/421:50:094
319359941070,84cyclictest0-21swapper/300:05:273
3193799408109,126cyclictest9585-21timerandwakeup22:35:245
319329940828,177cyclictest0-21swapper/000:19:340
319399940722,211cyclictest0-21swapper/722:29:587
319379940768,84cyclictest0-21swapper/522:29:595
319349940765,340cyclictest0-21swapper/221:50:052
319359940364,84cyclictest0-21swapper/320:29:323
319329940263,84cyclictest0-21swapper/022:35:240
319339940023,295cyclictest0-21swapper/123:55:031
319379939959,84cyclictest0-21swapper/520:00:035
319349939918,296cyclictest0-21swapper/200:20:092
319359939712,211cyclictest41-21ksoftirqd/300:19:323
319329939658,84cyclictest0-21swapper/020:57:520
319369939453,338cyclictest0-21swapper/421:00:104
319359939253,84cyclictest0-21swapper/323:55:013
319359939253,84cyclictest0-21swapper/300:37:563
319359939051,84cyclictest0-21swapper/322:29:543
319379938850,84cyclictest0-21swapper/500:19:545
319359938748,84cyclictest0-21swapper/321:00:023
319369938646,337cyclictest0-21swapper/400:37:564
3193499385129,126cyclictest33-21ksoftirqd/200:19:342
319399938443,338cyclictest0-21swapper/723:09:287
319359938144,84cyclictest0-21swapper/323:09:583
319339937578,84cyclictest0-21swapper/100:37:561
3193899372117,253cyclictest1955-21/usr/sbin/munin21:00:106
319329937233,84cyclictest0-21swapper/000:37:550
319389937132,84cyclictest0-21swapper/600:19:526
319369936928,338cyclictest0-21swapper/422:29:544
319329936869,126cyclictest121rcu_preempt22:30:000
319329936826,168cyclictest11060-21missed_timers00:20:180
3193899367111,253cyclictest0-21swapper/600:37:566
319359936728,84cyclictest0-21swapper/320:00:173
319349936627,84cyclictest0-21swapper/222:35:232
319399935820,45cyclictest0-21swapper/720:29:347
319379934994,84cyclictest0-21swapper/523:55:045
319399934545,126cyclictest32522-21snmpd21:30:107
31938993414,126cyclictest121rcu_preempt21:30:106
319349932670,84cyclictest33-21ksoftirqd/221:30:102
319369930912,42cyclictest0-21swapper/421:30:104
31933993056,210cyclictest0-21swapper/121:30:101
319329926055,79cyclictest121rcu_preempt21:30:010
31935992585,86cyclictest41-21ksoftirqd/321:30:003
319379925140,47cyclictest0-21swapper/521:30:105
3146229545,27sleep40-21swapper/419:09:244
3146028747,36sleep20-21swapper/219:09:232
3132028757,27sleep10-21swapper/119:07:211
3143228656,27sleep30-21swapper/319:09:003
321562830,0sleep10-21swapper/121:21:431
3126828251,8sleep50-21swapper/519:06:375
3137128150,28sleep70-21swapper/719:08:087
3130528151,27sleep60-21swapper/619:07:106
3122327947,28sleep00-21swapper/019:06:000
312642730,0sleep10-21swapper/123:04:131
312642730,0sleep10-21swapper/123:04:131
291162700,0sleep30-21swapper/323:02:473
291162700,0sleep30-21swapper/323:02:473
319422680,0sleep60-21swapper/621:55:076
264712650,2sleep43193699cyclictest23:17:324
85462540,0sleep40-21swapper/421:42:424
31933993911,16cyclictest25-21ksoftirqd/120:30:151
31933993911,16cyclictest25-21ksoftirqd/120:30:141
31932993815,13cyclictest0-21swapper/020:30:500
31932993815,13cyclictest0-21swapper/020:30:500
3193599345,21cyclictest41-21ksoftirqd/320:30:173
3193599345,21cyclictest41-21ksoftirqd/320:30:173
31938993327,3cyclictest0-21swapper/620:30:096
31938993327,3cyclictest0-21swapper/620:30:096
3193799332,3cyclictest0-21swapper/520:30:035
3193799332,3cyclictest0-21swapper/520:30:025
31934993223,6cyclictest17270-21sed20:30:172
31934993223,6cyclictest17270-21sed20:30:172
31939992921,5cyclictest18349-21cat20:30:297
31939992921,5cyclictest18349-21cat20:30:287
31939992910,18cyclictest0-21swapper/719:25:207
31938992910,18cyclictest0-21swapper/620:37:326
3193699291,2cyclictest0-21swapper/420:30:014
3193699291,2cyclictest0-21swapper/420:30:014
31936992910,18cyclictest0-21swapper/419:25:394
31933992910,18cyclictest0-21swapper/119:50:121
31933992910,18cyclictest0-21swapper/119:40:021
31933992910,18cyclictest0-21swapper/119:40:011
31932992910,18cyclictest0-21swapper/019:44:540
31932992910,18cyclictest0-21swapper/019:44:540
31939992828,0cyclictest0-21swapper/719:49:247
31939992810,18cyclictest0-21swapper/720:49:137
31939992810,18cyclictest0-21swapper/719:21:417
31939992810,18cyclictest0-21swapper/719:14:477
31939992810,0cyclictest0-21swapper/721:43:127
31938992810,18cyclictest0-21swapper/619:49:516
31938992810,18cyclictest0-21swapper/619:26:066
31938992810,18cyclictest0-21swapper/619:18:036
31937992810,17cyclictest0-21swapper/521:43:155
31935992828,0cyclictest0-21swapper/320:15:023
31935992810,18cyclictest0-21swapper/321:42:483
31935992810,18cyclictest0-21swapper/319:44:253
31935992810,18cyclictest0-21swapper/319:44:253
31935992810,18cyclictest0-21swapper/319:39:433
31935992810,18cyclictest0-21swapper/319:14:373
31935992810,0cyclictest0-21swapper/321:24:373
31935992810,0cyclictest0-21swapper/321:08:413
31935992810,0cyclictest0-21swapper/320:37:143
31935992810,0cyclictest0-21swapper/320:14:503
31935992810,0cyclictest0-21swapper/320:08:413
31935992810,0cyclictest0-21swapper/319:25:383
31934992810,18cyclictest0-21swapper/220:43:362
31933992810,18cyclictest0-21swapper/121:43:131
31933992810,17cyclictest0-21swapper/120:14:511
31933992810,17cyclictest0-21swapper/119:35:391
31933992810,0cyclictest0-21swapper/120:43:411
31933992810,0cyclictest0-21swapper/119:14:321
31932992810,18cyclictest0-21swapper/019:14:480
31932992810,17cyclictest0-21swapper/020:43:270
31932992810,0cyclictest0-21swapper/019:50:130
267882280,1sleep40-21swapper/421:35:214
31939992727,0cyclictest0-21swapper/721:24:367
31939992727,0cyclictest0-21swapper/720:15:027
31939992710,17cyclictest0-21swapper/720:43:317
31939992710,17cyclictest0-21swapper/720:36:517
31939992710,17cyclictest0-21swapper/720:14:217
31939992710,17cyclictest0-21swapper/720:08:407
31939992710,17cyclictest0-21swapper/719:50:127
31939992710,17cyclictest0-21swapper/719:35:217
31939992710,0cyclictest0-21swapper/721:08:387
31939992710,0cyclictest0-21swapper/719:40:057
31939992710,0cyclictest0-21swapper/719:40:047
31939992710,0cyclictest0-21swapper/719:17:547
31938992727,0cyclictest0-21swapper/620:08:366
31938992710,17cyclictest0-21swapper/621:42:406
31938992710,17cyclictest0-21swapper/621:24:426
31938992710,17cyclictest0-21swapper/621:08:226
31938992710,17cyclictest0-21swapper/619:21:486
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional