You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-21 - 19:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Mar 21, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2058499280,22cyclictest16297-21ssh11:50:041
2058499260,25cyclictest5723-21sh10:03:561
2059099250,1cyclictest14300-21diskmemload09:55:013
2058499251,24cyclictest0-21swapper/111:05:441
2058499251,23cyclictest29989-21ssh09:56:441
2058499250,24cyclictest14300-21diskmemload12:31:501
2058099250,1cyclictest14300-21diskmemload10:55:380
2059099240,23cyclictest2337-21munin-run11:05:003
20586992423,1cyclictest14300-21diskmemload10:26:272
2058699240,23cyclictest0-21swapper/209:56:172
20584992424,0cyclictest211rcuc/109:51:361
20584992423,0cyclictest0-21swapper/107:42:351
20584992422,1cyclictest13242-21diskstats11:47:361
20584992422,1cyclictest0-21swapper/112:22:401
20584992421,2cyclictest32452-21ssh11:33:481
2058499240,24cyclictest0-21swapper/112:26:361
2058499240,23cyclictest30563-21ssh12:36:551
2058499240,23cyclictest13191-21/usr/sbin/munin10:12:391
20580992424,0cyclictest19429-21apt-get09:15:020
20580992424,0cyclictest0-21swapper/012:33:530
2058099240,23cyclictest761-21sh09:59:130
2058099240,23cyclictest7261-21ssh10:37:410
2058099240,22cyclictest28050-21rm10:26:580
2058099240,0cyclictest14300-21diskmemload12:29:170
20590992323,0cyclictest14300-21diskmemload09:23:103
20590992323,0cyclictest0-21swapper/309:46:273
20590992323,0cyclictest0-21swapper/309:11:313
20590992322,1cyclictest14300-21diskmemload11:24:503
20590992322,1cyclictest14300-21diskmemload10:25:123
20590992322,0cyclictest0-21swapper/310:16:323
20590992321,1cyclictest29923-21turbostat08:35:013
20590992321,1cyclictest0-21swapper/307:45:013
20590992320,2cyclictest29881-21ssh12:35:523
2059099230,23cyclictest14300-21diskmemload11:36:133
2059099230,23cyclictest0-21swapper/310:52:003
2059099230,23cyclictest0-21swapper/310:42:503
2059099230,22cyclictest8419-21sh09:35:223
2059099230,22cyclictest7189-21rm10:05:553
2059099230,22cyclictest14300-21diskmemload09:42:383
2059099230,1cyclictest10561-21ssh12:17:143
2059099230,0cyclictest14300-21diskmemload10:38:493
2059099230,0cyclictest0-21swapper/312:33:583
2059099230,0cyclictest0-21swapper/312:08:573
2059099230,0cyclictest0-21swapper/311:10:033
2059099230,0cyclictest0-21swapper/309:30:133
2059099230,0cyclictest0-21swapper/309:16:543
20586992323,0cyclictest14300-21diskmemload11:07:442
20586992321,2cyclictest7162-21ssh11:41:412
20586992321,1cyclictest23801-21ssh11:25:432
20586992321,1cyclictest14300-21diskmemload12:19:312
20586992321,1cyclictest0-21swapper/210:12:452
20586992318,4cyclictest0-21swapper/211:04:372
2058699230,2cyclictest22688-21sh09:49:072
2058699230,23cyclictest14300-21diskmemload12:27:412
2058699230,22cyclictest3091-21ssh12:08:472
2058699230,22cyclictest27707-21ssh12:02:072
2058699230,22cyclictest18292-21cat11:52:352
2058699230,1cyclictest611-21irqbalance11:55:262
2058699230,0cyclictest0-21swapper/210:02:362
2058699230,0cyclictest0-21swapper/209:37:122
2058699230,0cyclictest0-21swapper/209:21:392
20584992323,0cyclictest14300-21diskmemload11:55:321
20584992323,0cyclictest0-21swapper/111:10:521
20584992323,0cyclictest0-21swapper/109:18:051
20584992322,1cyclictest14300-21diskmemload12:10:271
20584992321,2cyclictest32506-21ssh10:31:061
20584992321,1cyclictest25478-21ssh10:55:571
20584992321,1cyclictest18702-21sh11:20:401
20584992321,1cyclictest14300-21diskmemload09:44:011
20584992321,1cyclictest0-21swapper/111:36:111
20584992321,1cyclictest0-21swapper/111:01:081
20584992321,1cyclictest0-21swapper/110:26:541
20584992321,1cyclictest0-21swapper/109:37:481
20584992320,2cyclictest774-21runrttasks09:11:561
20584992320,2cyclictest14300-21diskmemload09:47:411
2058499230,2cyclictest25138-21munin-run08:25:011
2058499230,23cyclictest14300-21diskmemload10:05:311
2058499230,23cyclictest0-21swapper/112:16:211
2058499230,23cyclictest0-21swapper/110:15:171
2058499230,22cyclictest9713-21ssh10:40:171
2058499230,22cyclictest23569-21ssh11:25:211
2058499230,22cyclictest0-21swapper/110:48:201
2058499230,22cyclictest0-21swapper/109:27:131
2058499230,1cyclictest26485-21sh09:22:181
20580992323,0cyclictest0-21swapper/011:41:560
20580992323,0cyclictest0-21swapper/011:14:350
20580992323,0cyclictest0-21swapper/011:07:140
20580992322,1cyclictest24717-21sh11:27:150
20580992322,1cyclictest0-21swapper/009:12:160
20580992321,2cyclictest20193-21rm10:17:520
20580992320,2cyclictest8838-21sh09:36:010
20580992320,2cyclictest22139-21ssh11:56:180
20580992320,2cyclictest0-21swapper/011:30:550
2058099230,2cyclictest17636-21rm11:52:120
2058099230,23cyclictest14300-21diskmemload12:24:090
2058099230,23cyclictest0-21swapper/012:15:110
2058099230,23cyclictest0-21swapper/011:39:490
2058099230,23cyclictest0-21swapper/011:19:230
2058099230,22cyclictest4199-21sh10:34:590
2058099230,22cyclictest31886-21ssh09:27:070
2058099230,22cyclictest29843-21sh12:35:480
2058099230,22cyclictest23605-21cstates10:22:350
2058099230,22cyclictest14331-21perf09:10:010
2058099230,22cyclictest14300-21diskmemload11:00:330
2058099230,22cyclictest14300-21diskmemload09:50:260
2058099230,22cyclictest0-21swapper/010:45:380
2058099230,22cyclictest0-21swapper/010:10:520
2058099230,1cyclictest14300-21diskmemload12:12:020
2058099230,1cyclictest0-21swapper/010:03:150
2058099230,0cyclictest0-21swapper/011:46:160
20590992222,0cyclictest14300-21diskmemload10:00:253
20590992222,0cyclictest0-21swapper/311:15:563
20590992222,0cyclictest0-21swapper/310:30:023
20590992222,0cyclictest0-21swapper/310:13:073
20590992220,1cyclictest17917-21apt-get11:52:343
20590992219,2cyclictest26565-21ssh12:00:143
2059099220,21cyclictest6663-21ssh11:40:563
2059099220,21cyclictest28045-21ssh10:57:433
2059099220,21cyclictest14300-21diskmemload10:20:153
2059099220,21cyclictest11458-21ssh11:45:243
2059099220,21cyclictest0-21swapper/311:55:023
2059099220,21cyclictest0-21swapper/311:30:063
2059099220,21cyclictest0-21swapper/310:45:253
2059099220,1cyclictest4715-21apt08:50:003
2059099220,1cyclictest2586-21ssh11:05:033
2059099220,1cyclictest0-21swapper/312:23:243
2059099220,1cyclictest0-21swapper/309:55:473
20586992222,0cyclictest14300-21diskmemload12:36:352
20586992222,0cyclictest14300-21diskmemload09:50:192
20586992221,1cyclictest3516-21ssh09:30:322
20586992221,1cyclictest14300-21diskmemload09:11:272
20586992220,1cyclictest8723-21sh12:14:412
20586992220,1cyclictest24637-21ssh12:30:262
20586992220,1cyclictest14293-21cron09:10:012
20586992220,1cyclictest0-21swapper/210:15:032
2058699220,22cyclictest4812-21sh11:38:172
2058699220,22cyclictest14473-21ssh09:41:542
2058699220,22cyclictest0-21swapper/211:23:252
2058699220,22cyclictest0-21swapper/211:15:342
2058699220,22cyclictest0-21swapper/210:51:132
2058699220,22cyclictest0-21swapper/209:25:282
2058699220,21cyclictest5876-21perf07:45:012
2058699220,21cyclictest5172-21apt-get08:52:332
2058699220,21cyclictest14300-21diskmemload10:35:032
2058699220,21cyclictest0-21swapper/210:41:402
2058699220,21cyclictest0-21swapper/210:20:022
2058699220,1cyclictest0-21swapper/211:13:422
2058699220,1cyclictest0-21swapper/210:57:162
2058699220,0cyclictest0-21swapper/211:49:532
20584992222,0cyclictest14300-21diskmemload12:01:261
20584992220,1cyclictest0-21swapper/108:27:351
2058499221,0cyclictest0-21swapper/110:36:251
2058499220,21cyclictest31418-21apt-get07:32:341
2058499220,21cyclictest14300-21diskmemload11:15:401
2058499220,21cyclictest1389-21apt-get07:37:341
2058499220,21cyclictest0-21swapper/110:50:071
2058499220,21cyclictest0-21swapper/109:30:301
2058499220,21cyclictest0-21swapper/107:12:331
2058499220,1cyclictest2085-21ssh12:07:411
20580992222,0cyclictest0-21swapper/010:53:160
20580992222,0cyclictest0-21swapper/010:05:570
20580992220,2cyclictest0-21swapper/009:21:170
20580992220,1cyclictest10647-21sh07:55:000
20580992220,1cyclictest0-21swapper/009:46:440
2058099221,21cyclictest0-21swapper/012:05:250
2058099220,21cyclictest5876-21perf07:45:010
2058099220,21cyclictest3253-21ssh09:30:080
2058099220,21cyclictest0-21swapper/012:00:220
2058099220,21cyclictest0-21swapper/011:22:350
2058099220,21cyclictest0-21swapper/010:40:270
2058099220,21cyclictest0-21swapper/008:15:010
2058099220,1cyclictest0-21swapper/009:42:060
20590992119,1cyclictest11912-21cron09:05:013
20590992119,1cyclictest0-21swapper/308:40:003
2059099210,21cyclictest0-21swapper/312:25:043
2059099210,21cyclictest0-21swapper/311:26:293
2059099210,20cyclictest28165-21apt-get07:25:023
2059099210,20cyclictest15970-21apt-get08:07:333
2059099210,20cyclictest0-21swapper/307:32:343
2059099210,1cyclictest4291-21ssh12:10:223
2059099210,1cyclictest0-21swapper/307:50:003
2059099210,0cyclictest0-21swapper/307:57:343
20586992121,0cyclictest0-21swapper/210:30:092
20586992121,0cyclictest0-21swapper/210:05:292
2058699210,21cyclictest0-21swapper/211:31:322
2058699210,21cyclictest0-21swapper/210:46:112
2058699210,21cyclictest0-21swapper/208:47:342
2058699210,1cyclictest30972-21munin-run07:30:012
2058699210,0cyclictest27978-21apt-get08:32:352
2058699210,0cyclictest0-21swapper/209:15:192
20584992117,3cyclictest3220-21cut08:47:361
2058499210,21cyclictest0-21swapper/110:20:051
2058499210,0cyclictest29016-21apt-get07:27:321
2058499210,0cyclictest0-21swapper/108:52:331
20580992119,1cyclictest0-21swapper/008:50:000
20590992020,0cyclictest0-21swapper/309:07:333
20590992020,0cyclictest0-21swapper/308:52:333
20590992020,0cyclictest0-21swapper/308:12:343
20590992020,0cyclictest0-21swapper/307:17:343
20590992018,1cyclictest22972-21latency_hist07:15:003
2059099200,20cyclictest0-21swapper/307:22:353
2059099200,0cyclictest25581-21apt-get08:27:353
20586992020,0cyclictest13523-21apt-get08:02:342
20586992020,0cyclictest0-21swapper/209:02:342
2058699200,20cyclictest0-21swapper/208:37:362
2058699200,1cyclictest3473-21munin-run07:40:012
20584992020,0cyclictest20771-21apt-get08:17:351
20584992020,0cyclictest0-21swapper/108:02:331
20584992020,0cyclictest0-21swapper/107:17:341
20584992019,1cyclictest29937-21munin-run08:35:011
2058499200,0cyclictest0-21swapper/108:42:351
20580992020,0cyclictest0-21swapper/009:02:340
20580992020,0cyclictest0-21swapper/008:57:340
20580992018,1cyclictest28165-21apt-get07:25:010
2058099200,20cyclictest0-21swapper/008:42:340
2058099200,0cyclictest0-21swapper/008:02:340
20586991919,0cyclictest0-21swapper/207:57:342
20586991918,1cyclictest21018-21apt-get07:12:332
20586991916,2cyclictest313-21systemd-journal08:45:002
2058699190,18cyclictest0-21swapper/208:07:322
2058499190,18cyclictest0-21swapper/108:15:011
20580991919,0cyclictest8261-21latency_hist07:50:000
20586991818,0cyclictest26229-21apt-get07:22:352
20586991818,0cyclictest0-21swapper/208:17:352
20586991817,0cyclictest0-21swapper/207:35:012
2058699180,0cyclictest0-21swapper/208:12:342
20584991818,0cyclictest30377-21apt-get08:37:351
20584991818,0cyclictest0-21swapper/109:07:331
2058499181,2cyclictest9563-21sed09:00:011
20580991815,2cyclictest3484-21idleruntime-cro07:40:010
2059099172,14cyclictest22688-21cron08:20:013
20590991717,0cyclictest0-21swapper/308:57:343
20590991717,0cyclictest0-21swapper/307:52:343
20590991717,0cyclictest0-21swapper/307:37:343
20590991715,1cyclictest606-21dbus-daemon08:45:003
20584991715,1cyclictest0-21swapper/107:47:351
20580991717,0cyclictest0-21swapper/008:32:350
20586991616,0cyclictest0-21swapper/208:22:332
2058699160,15cyclictest9720-21munin-run09:00:012
20584991616,0cyclictest0-21swapper/107:57:341
20580991616,0cyclictest0-21swapper/008:27:350
20580991614,1cyclictest23598-21sshd07:15:170
20580991614,1cyclictest0-21swapper/007:15:000
20586991515,0cyclictest0-21swapper/208:27:352
20586991515,0cyclictest0-21swapper/207:52:342
20586991515,0cyclictest0-21swapper/207:47:352
20584991515,0cyclictest0-21swapper/109:02:421
20584991515,0cyclictest0-21swapper/107:22:351
20580991514,1cyclictest0-21swapper/007:29:290
20580991513,1cyclictest11106-21apt-get07:57:340
2058099150,1cyclictest0-21swapper/008:37:360
2058099150,14cyclictest0-21swapper/007:30:400
2059099140,14cyclictest0-21swapper/308:20:303
2059099140,14cyclictest0-21swapper/308:00:133
20586991414,0cyclictest0-21swapper/207:15:352
2058499140,0cyclictest0-21swapper/108:05:151
20580991414,0cyclictest0-21swapper/008:15:410
2058099140,14cyclictest0-21swapper/008:50:110
2058099140,14cyclictest0-21swapper/008:05:300
2058099140,0cyclictest0-21swapper/008:20:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional