You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-15 - 07:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Wed Jul 15, 2026 00:56:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
212699975270,84cyclictest0-21swapper/223:49:062
212699975071,84cyclictest0-21swapper/222:43:232
212699971477,634cyclictest26225-21ssh00:42:012
212719969319,387cyclictest0-21swapper/400:32:204
212699967281,176cyclictest2500-21idleruntime-cro23:35:022
212699967281,176cyclictest2500-21idleruntime-cro23:35:022
212699967281,176cyclictest2500-21idleruntime-cro23:35:012
212699967281,176cyclictest2500-21idleruntime-cro23:35:012
2127399660106,381cyclictest0-21swapper/600:32:076
2126999655400,253cyclictest0-21swapper/200:32:012
212689964494,126cyclictest0-21swapper/123:08:071
21274996428,2cyclictest0-21swapper/700:27:217
2127099639129,507cyclictest0-21swapper/323:52:483
212679963990,334cyclictest0-21swapper/022:24:420
212749963683,550cyclictest0-21swapper/723:52:397
2127299636591,42cyclictest551rcuc/520:53:095
21267996350,84cyclictest0-21swapper/000:32:010
212709963380,380cyclictest121rcu_preempt22:24:343
212679963381,126cyclictest0-21swapper/020:55:170
212699962875,550cyclictest0-21swapper/223:17:402
2127099627158,338cyclictest29074-21diskmemload00:14:193
212719962230,42cyclictest0-21swapper/423:19:404
2126999622112,507cyclictest0-21swapper/222:23:432
2126799621110,380cyclictest0-21swapper/021:40:200
2127399615147,295cyclictest0-21swapper/622:42:386
2126999611103,126cyclictest14791-21kworker/u16:021:40:202
212689961060,211cyclictest21265-21cyclictest00:32:121
212699960812,507cyclictest0-21swapper/223:50:242
2126799608101,126cyclictest0-21swapper/023:52:480
212689960615,253cyclictest0-21swapper/123:35:291
212719960510,2cyclictest0-21swapper/422:02:154
21269996027,592cyclictest0-21swapper/223:09:242
212689960211,2cyclictest0-21swapper/100:27:201
21272996008,126cyclictest0-21swapper/523:34:465
21272996008,126cyclictest0-21swapper/523:34:465
21272996008,126cyclictest0-21swapper/523:34:465
21272996008,126cyclictest0-21swapper/523:34:465
212689960046,381cyclictest0-21swapper/121:54:011
21270995964,2cyclictest0-21swapper/300:27:213
212739958535,83cyclictest121rcu_preempt21:53:176
212729958372,211cyclictest561ktimersoftd/500:15:305
212719958334,84cyclictest0-21swapper/422:22:594
212699958171,84cyclictest18543-21ssh21:36:472
212679958069,211cyclictest0-21swapper/022:59:420
2126899577152,211cyclictest29074-21diskmemload22:23:331
212699957220,127cyclictest0-21swapper/200:26:582
2127399570313,126cyclictest0-21swapper/623:52:306
212719956916,337cyclictest0-21swapper/423:34:334
212719956916,337cyclictest0-21swapper/423:34:334
212719956916,337cyclictest0-21swapper/423:34:334
212719956916,337cyclictest0-21swapper/423:34:334
2127499566480,84cyclictest711rcuc/723:55:077
212679956612,381cyclictest0-21swapper/020:53:590
212699956512,465cyclictest0-21swapper/223:35:132
2127299562221,84cyclictest30351-1kworker/u17:122:40:245
2127499561224,207cyclictest25046-1kworker/u17:123:00:087
2127099560515,42cyclictest401ktimersoftd/321:39:003
21267995588,338cyclictest0-21swapper/023:34:580
21267995588,338cyclictest0-21swapper/023:34:580
21267995588,338cyclictest0-21swapper/023:34:580
21267995588,338cyclictest0-21swapper/023:34:580
212679955810,7cyclictest121rcu_preempt22:03:480
2127399557130,254cyclictest0-21swapper/623:08:076
21269995563,550cyclictest0-21swapper/222:05:242
21273995551,297cyclictest0-21swapper/623:55:076
21269995521,85cyclictest0-21swapper/221:54:012
212699955080,296cyclictest0-21swapper/222:02:082
21268995500,296cyclictest121rcu_preempt21:30:051
21268995500,296cyclictest121rcu_preempt21:30:051
21268995500,296cyclictest121rcu_preempt21:30:051
21271995491,126cyclictest0-21swapper/423:05:594
212709954980,338cyclictest22555-21fschecks_time21:40:253
212689954838,507cyclictest0-21swapper/123:52:481
2127299546119,296cyclictest0-21swapper/500:26:575
212689954678,295cyclictest0-21swapper/121:38:501
212689954676,296cyclictest0-21swapper/123:48:471
212689954576,338cyclictest121rcu_preempt20:52:071
212689954531,126cyclictest0-21swapper/122:44:021
2127299543288,253cyclictest561ktimersoftd/523:10:025
2126799543248,84cyclictest3863-21cut23:35:200
212719954277,126cyclictest0-21swapper/400:14:444
212719954232,126cyclictest0-21swapper/422:56:594
212719954133,90cyclictest0-21swapper/421:28:024
212719954133,90cyclictest0-21swapper/421:28:024
212719954133,90cyclictest0-21swapper/421:28:024
212719954133,90cyclictest0-21swapper/421:28:014
212719954133,84cyclictest0-21swapper/423:51:374
2127399540198,339cyclictest21265-21cyclictest00:01:326
212729953970,338cyclictest0-21swapper/520:55:175
212679953969,169cyclictest9-21ksoftirqd/021:36:130
2127499537240,295cyclictest10920-1kworker/u17:122:21:177
2126799537110,211cyclictest17279-21ssh00:01:520
212739953628,84cyclictest0-21swapper/621:29:026
212739953628,84cyclictest0-21swapper/621:29:026
212739953628,84cyclictest0-21swapper/621:29:026
212739953628,84cyclictest0-21swapper/621:29:016
212689953629,126cyclictest0-21swapper/100:41:171
2126799536111,168cyclictest9-21ksoftirqd/022:40:540
212699953524,295cyclictest0-21swapper/220:54:512
212699953428,338cyclictest0-21swapper/221:30:052
212699953428,338cyclictest0-21swapper/221:30:052
212699953428,338cyclictest0-21swapper/221:30:042
2127199532193,84cyclictest64450irq/122-eth023:10:224
212709953161,296cyclictest0-21swapper/322:05:243
2127099530106,211cyclictest41-21ksoftirqd/323:07:563
2127099530102,296cyclictest18730-1kworker/u17:200:15:293
2126899530147,126cyclictest121rcu_preempt23:55:111
212699952863,169cyclictest3917-21ssh00:11:322
2127399527312,127cyclictest0-21swapper/622:02:126
212679952719,84cyclictest0-21swapper/022:05:240
212739952658,465cyclictest0-21swapper/623:10:026
2127399526400,42cyclictest631rcuc/600:13:436
212719952518,126cyclictest0-21swapper/423:00:174
212739952417,380cyclictest0-21swapper/623:17:336
212709952417,338cyclictest0-21swapper/323:35:283
212679952416,89cyclictest0-21swapper/023:00:150
2127199523182,338cyclictest22058-21nscd21:53:074
2127299521180,42cyclictest29074-21diskmemload23:09:535
212709952113,126cyclictest3492-21ssh21:29:463
212709952113,126cyclictest3492-21ssh21:29:463
212709952113,126cyclictest3492-21ssh21:29:463
212709952113,126cyclictest3492-21ssh21:29:463
212729952012,126cyclictest0-21swapper/523:52:485
21269995209,508cyclictest0-21swapper/222:57:302
21269995184,423cyclictest704-21nscd00:01:082
21273995168,211cyclictest0-21swapper/621:40:026
2127399516432,3cyclictest631rcuc/622:24:506
21271995168,126cyclictest21192-21kworker/u16:300:41:344
21268995169,126cyclictest0-21swapper/100:15:281
2127199513129,169cyclictest49-21ksoftirqd/421:36:124
21269995125,338cyclictest0-21swapper/221:28:412
21267995115,2cyclictest0-21swapper/000:27:210
212739951044,127cyclictest0-21swapper/620:54:516
21274995090,84cyclictest0-21swapper/723:08:407
21272995090,84cyclictest0-21swapper/521:29:415
21272995090,84cyclictest0-21swapper/521:29:415
21272995090,84cyclictest0-21swapper/521:29:415
21272995090,84cyclictest0-21swapper/521:29:415
21267995090,84cyclictest27651-21ssh23:48:050
2127199508210,126cyclictest121rcu_preempt22:43:314
212679950882,211cyclictest9-21ksoftirqd/023:07:560
2127299507124,380cyclictest0-21swapper/523:35:465
212729950638,295cyclictest0-21swapper/523:00:225
212719950679,211cyclictest481ktimersoftd/400:01:504
2127099505422,81cyclictest391rcuc/322:43:303
2127499504247,254cyclictest721ktimersoftd/723:19:547
2126799504165,84cyclictest9686-21kworker/u16:123:17:240
2127399503248,253cyclictest641ktimersoftd/621:40:126
212739950275,296cyclictest21265-21cyclictest23:00:176
212739950136,127cyclictest0-21swapper/600:41:526
212719950138,261cyclictest1551-21snmpd23:47:504
212689950032,211cyclictest5778-21rm23:00:091
2126999499455,42cyclictest311rcuc/222:25:042
212679949931,257cyclictest18730-1kworker/u17:200:14:370
212679949876,84cyclictest0-21swapper/000:15:380
212679949830,295cyclictest0-21swapper/000:41:070
212749949733,380cyclictest0-21swapper/723:35:217
212689949732,126cyclictest2532-21basename23:35:021
212689949732,126cyclictest2532-21basename23:35:021
212689949732,126cyclictest2532-21basename23:35:021
212689949732,126cyclictest2532-21basename23:35:021
2127499496241,252cyclictest20759-21kworker/u16:221:54:007
2127399496370,42cyclictest641ktimersoftd/600:27:386
212689949569,211cyclictest25-21ksoftirqd/123:17:111
212739949430,382cyclictest641ktimersoftd/600:15:186
2127499492192,127cyclictest121rcu_preempt00:14:447
2126899490149,126cyclictest121rcu_preempt22:04:431
212689948824,381cyclictest0-21swapper/100:13:541
212699948721,126cyclictest0-21swapper/221:30:202
212699948718,466cyclictest0-21swapper/222:45:132
212749948562,84cyclictest121rcu_preempt23:35:057
212749948562,84cyclictest121rcu_preempt23:35:057
212749948562,84cyclictest121rcu_preempt23:35:057
212749948562,84cyclictest121rcu_preempt23:35:057
212749948521,380cyclictest0-21swapper/721:37:507
212719948516,211cyclictest0-21swapper/400:27:404
212719948457,296cyclictest0-21swapper/420:52:474
212729948360,169cyclictest0-21swapper/522:21:485
212679948318,83cyclictest0-21swapper/021:53:510
212729948213,296cyclictest29074-21diskmemload21:53:395
212709948113,211cyclictest0-21swapper/300:41:323
212749948014,84cyclictest0-21swapper/722:59:577
212729947914,84cyclictest0-21swapper/523:47:415
2126799478137,126cyclictest121rcu_preempt00:35:280
21274994737,84cyclictest0-21swapper/723:48:477
21274994715,211cyclictest0-21swapper/721:40:127
21272994715,85cyclictest0-21swapper/500:41:335
2127299469128,338cyclictest21265-21cyclictest22:02:575
21273994680,3cyclictest0-21swapper/622:45:136
212749946744,4cyclictest1553-21runrttasks21:29:177
212749946744,4cyclictest1553-21runrttasks21:29:177
212749946744,4cyclictest1553-21runrttasks21:29:177
212749946744,4cyclictest1553-21runrttasks21:29:177
2127399467300,41cyclictest0-21swapper/620:55:236
21272994671,211cyclictest21265-21cyclictest23:19:115
212749946681,211cyclictest73-21ksoftirqd/700:01:327
21273994661,126cyclictest0-21swapper/623:34:316
21273994661,126cyclictest0-21swapper/623:34:316
21273994661,126cyclictest0-21swapper/623:34:306
21273994661,126cyclictest0-21swapper/623:34:306
21271994661,45cyclictest0-21swapper/420:55:284
2127199465171,48cyclictest121rcu_preempt00:15:374
2127199465124,338cyclictest0-21swapper/422:25:154
2127499464123,338cyclictest0-21swapper/722:02:537
2127099464123,84cyclictest41-21ksoftirqd/323:34:193
2127099464123,84cyclictest41-21ksoftirqd/323:34:193
2127099464123,84cyclictest41-21ksoftirqd/323:34:193
2127099464123,84cyclictest41-21ksoftirqd/323:34:193
212699946438,423cyclictest0-21swapper/223:20:092
212709946340,256cyclictest0-21swapper/321:52:553
2126899463121,126cyclictest121rcu_preempt21:40:261
212679946379,211cyclictest2910-21ssh21:29:020
212679946379,211cyclictest2910-21ssh21:29:020
212679946379,211cyclictest2910-21ssh21:29:020
212679946379,211cyclictest2910-21ssh21:29:010
212729946282,84cyclictest0-21swapper/522:59:025
212709946180,84cyclictest121rcu_preempt23:47:193
212739946034,84cyclictest0-21swapper/623:48:056
212689946036,127cyclictest3449-21ssh22:57:471
212689946034,338cyclictest0-21swapper/121:27:461
2127299459160,126cyclictest121rcu_preempt21:37:355
212719945934,84cyclictest0-21swapper/423:20:094
2127199459118,338cyclictest3792-21ssh23:35:174
212709945975,381cyclictest0-21swapper/323:16:503
212739945876,380cyclictest0-21swapper/623:35:466
2127499457116,338cyclictest0-21swapper/722:43:597
212709945734,257cyclictest0-21swapper/322:04:303
212689945735,337cyclictest0-21swapper/100:35:141
2127099456413,41cyclictest401ktimersoftd/322:58:233
212749945574,126cyclictest73-21ksoftirqd/720:55:267
212689945533,256cyclictest0-21swapper/120:55:211
2127499454198,253cyclictest27385-1kworker/u17:222:05:197
212709945432,208cyclictest0-21swapper/320:51:503
212679945331,261cyclictest698-21ptp4l23:10:030
212719945273,84cyclictest0-21swapper/421:40:154
212749945129,256cyclictest0-21swapper/700:32:317
212699945123,127cyclictest9994-21ssh00:15:322
212689945125,295cyclictest0-21swapper/123:10:191
212699944927,2cyclictest0-21swapper/220:55:212
212679944726,84cyclictest0-21swapper/021:30:150
212739944622,211cyclictest0-21swapper/622:59:576
2127299445107,83cyclictest121rcu_preempt00:32:285
212749944422,338cyclictest0-21swapper/700:15:187
212709944460,169cyclictest41-21ksoftirqd/320:55:173
212709944220,2cyclictest0-21swapper/323:00:083
2127099442102,338cyclictest0-21swapper/322:25:143
212679944119,167cyclictest121rcu_preempt22:45:120
2126799441101,338cyclictest2974-21ssh23:55:080
212709943956,210cyclictest0-21swapper/323:55:083
212699943913,126cyclictest2960-21sh23:55:022
212719943816,84cyclictest0-21swapper/422:05:144
212739943755,126cyclictest121rcu_preempt00:35:146
2127399437350,84cyclictest631rcuc/621:30:156
212749943614,84cyclictest0-21swapper/700:41:077
212729943655,84cyclictest0-21swapper/522:05:175
212709943656,256cyclictest698-21ptp4l00:32:203
212739943311,84cyclictest0-21swapper/622:05:156
212719943355,83cyclictest121rcu_preempt00:35:374
21268994317,84cyclictest0-21swapper/122:05:081
212729943091,84cyclictest0-21swapper/523:20:105
212749942742,211cyclictest0-21swapper/720:53:297
212729942482,211cyclictest57-21ksoftirqd/500:01:085
212679942382,338cyclictest0-21swapper/022:25:040
212749942282,338cyclictest0-21swapper/700:35:277
212739942280,339cyclictest0-21swapper/621:55:126
212729942282,209cyclictest0-21swapper/500:14:035
212729942281,126cyclictest22367-21/usr/sbin/munin21:40:205
212709942282,338cyclictest0-21swapper/300:01:233
212699942282,338cyclictest0-21swapper/200:35:272
212709942140,296cyclictest0-21swapper/300:35:103
212689942181,338cyclictest0-21swapper/100:01:361
212719942079,338cyclictest64450irq/122-eth022:45:124
212709942040,296cyclictest0-21swapper/322:45:103
212689942040,295cyclictest0-21swapper/122:25:051
212729941979,84cyclictest0-21swapper/521:30:165
212709941939,48cyclictest0-21swapper/323:10:223
212699941838,296cyclictest0-21swapper/223:10:192
212689941838,295cyclictest0-21swapper/121:30:201
212699941778,336cyclictest0-21swapper/223:00:242
212689941777,338cyclictest0-21swapper/122:45:121
212679941574,338cyclictest0-21swapper/021:55:120
212709941474,338cyclictest0-21swapper/321:30:163
212679941335,211cyclictest7602-21sh23:20:100
212749941169,339cyclictest0-21swapper/723:10:017
2127499410111,126cyclictest0-21swapper/721:30:217
212729941071,84cyclictest0-21swapper/522:25:035
212719940968,338cyclictest0-21swapper/423:55:064
212699940970,84cyclictest0-21swapper/221:55:102
212749940866,339cyclictest0-21swapper/722:45:127
212739940726,85cyclictest0-21swapper/622:25:046
212709940727,296cyclictest0-21swapper/321:55:023
212689940666,337cyclictest0-21swapper/123:20:171
212749940363,338cyclictest0-21swapper/722:25:127
212719940363,337cyclictest0-21swapper/421:55:104
2127299401146,125cyclictest121rcu_preempt22:45:105
212719940122,260cyclictest0-21swapper/421:30:194
212739939212,84cyclictest121rcu_preempt23:20:106
212689938848,338cyclictest0-21swapper/100:20:331
212749938746,338cyclictest7590-21sh23:20:077
212709938039,338cyclictest0-21swapper/300:20:343
212709938039,211cyclictest0-21swapper/323:20:063
212729937882,84cyclictest0-21swapper/500:20:355
2127299378122,253cyclictest17347-21ssh21:55:125
2126899376122,84cyclictest0-21swapper/121:55:121
212729937435,84cyclictest0-21swapper/500:35:235
212679936725,212cyclictest30705-21kworker/u16:000:20:340
212749936527,171cyclictest0-21swapper/700:20:357
212739936324,84cyclictest0-21swapper/600:20:356
212719936023,3cyclictest0-21swapper/400:20:354
212749935516,255cyclictest0-21swapper/721:55:027
2127299354144,44cyclictest121rcu_preempt23:55:085
212699935053,84cyclictest0-21swapper/200:20:362
2074329545,28sleep40-21swapper/419:08:374
310002920,1sleep30-21swapper/321:45:013
2080129064,4sleep50-21swapper/519:09:265
2083428556,26sleep20-21swapper/219:09:562
2057528555,27sleep70-21swapper/719:06:167
2060928446,35sleep30-21swapper/319:06:433
2069728252,27sleep60-21swapper/619:07:596
2077627953,23sleep00-21swapper/019:09:050
2056927849,26sleep10-21swapper/119:06:101
21269993310,0cyclictest0-21swapper/222:39:502
21269993226,5cyclictest0-21swapper/221:14:052
21269993125,5cyclictest0-21swapper/219:59:412
21271992910,18cyclictest0-21swapper/422:18:504
21271992910,18cyclictest0-21swapper/422:18:504
21274992810,18cyclictest0-21swapper/722:18:377
21274992810,18cyclictest0-21swapper/722:18:367
21274992810,18cyclictest0-21swapper/720:11:237
21274992810,0cyclictest0-21swapper/722:39:387
21274992810,0cyclictest0-21swapper/721:14:027
21274992810,0cyclictest0-21swapper/720:37:537
21274992810,0cyclictest0-21swapper/719:59:407
21273992828,0cyclictest0-21swapper/619:48:406
21273992810,18cyclictest0-21swapper/619:12:466
21273992810,18cyclictest0-21swapper/619:12:466
21273992810,17cyclictest0-21swapper/622:18:216
21273992810,17cyclictest0-21swapper/622:18:216
21272992810,17cyclictest0-21swapper/521:13:265
21271992810,17cyclictest0-21swapper/420:37:174
21271992810,17cyclictest0-21swapper/419:48:434
21271992810,17cyclictest0-21swapper/419:12:454
21271992810,17cyclictest0-21swapper/419:12:454
21270992810,18cyclictest0-21swapper/322:18:493
21270992810,18cyclictest0-21swapper/322:18:493
21270992810,18cyclictest0-21swapper/321:13:183
21270992810,18cyclictest0-21swapper/320:37:283
21270992810,18cyclictest0-21swapper/320:25:033
21270992810,18cyclictest0-21swapper/319:31:563
21270992810,18cyclictest0-21swapper/319:12:453
21270992810,18cyclictest0-21swapper/319:12:453
21270992810,0cyclictest0-21swapper/322:39:413
21270992810,0cyclictest0-21swapper/320:24:313
21270992810,0cyclictest0-21swapper/320:11:383
21268992810,18cyclictest0-21swapper/121:13:331
21268992810,0cyclictest0-21swapper/122:39:181
21274992727,0cyclictest0-21swapper/719:12:467
21274992727,0cyclictest0-21swapper/719:12:457
21274992710,17cyclictest0-21swapper/720:24:277
21274992710,17cyclictest0-21swapper/719:38:217
21274992710,17cyclictest0-21swapper/719:32:047
21274992710,0cyclictest0-21swapper/719:48:407
21273992727,0cyclictest0-21swapper/622:39:186
21273992710,17cyclictest0-21swapper/621:13:196
21273992710,17cyclictest0-21swapper/620:37:276
21273992710,17cyclictest0-21swapper/620:25:016
21273992710,17cyclictest0-21swapper/620:24:276
21273992710,17cyclictest0-21swapper/620:10:586
21273992710,17cyclictest0-21swapper/619:38:246
21273992710,17cyclictest0-21swapper/619:31:586
21273992710,0cyclictest0-21swapper/619:59:136
21272992727,0cyclictest0-21swapper/520:11:195
21272992710,17cyclictest0-21swapper/522:18:285
21272992710,17cyclictest0-21swapper/522:18:285
21272992710,17cyclictest0-21swapper/520:37:195
21272992710,17cyclictest0-21swapper/520:25:015
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional