You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-21 - 07:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Sat Mar 21, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
933499280,25cyclictest0-21swapper/121:37:301
9340992623,3cyclictest2986-21diskmemload23:55:213
9340992620,5cyclictest0-21swapper/300:32:313
934099260,2cyclictest4710-21sh21:12:193
9340992520,4cyclictest2986-21diskmemload23:12:323
9340992520,4cyclictest2833-21ssh23:17:123
9340992519,5cyclictest3137-21users22:12:413
934099250,24cyclictest17293-21ssh22:58:503
934099250,21cyclictest24308-21ssh23:06:353
934099250,21cyclictest18205-21ssh23:32:293
934099250,1cyclictest0-21swapper/321:38:393
9336992522,2cyclictest9105-21rm21:16:102
933699250,24cyclictest0-21swapper/200:19:462
933699250,21cyclictest0-21swapper/223:15:332
933499250,1cyclictest0-21swapper/123:58:391
932899250,24cyclictest0-21swapper/020:22:310
932899250,24cyclictest0-21swapper/000:24:430
932899250,0cyclictest2986-21diskmemload23:15:050
9340992423,1cyclictest2986-21diskmemload22:09:483
9340992421,3cyclictest0-21swapper/321:28:343
9340992420,3cyclictest5072-21sh21:43:213
9340992420,3cyclictest2986-21diskmemload23:00:303
9340992420,3cyclictest2986-21diskmemload00:26:103
934099240,23cyclictest0-21swapper/321:55:353
934099240,23cyclictest0-21swapper/321:48:103
934099240,20cyclictest0-21swapper/321:17:033
934099240,0cyclictest27764-21ssh21:34:063
934099240,0cyclictest0-21swapper/323:36:593
934099240,0cyclictest0-21swapper/320:12:323
934099240,0cyclictest0-21swapper/300:21:333
9336992422,1cyclictest19629-21ssh00:36:502
9336992421,2cyclictest29359-21ssh21:36:152
933699240,23cyclictest24104-21ssh21:31:002
933699240,1cyclictest24831-21diskmemload00:09:172
9334992424,0cyclictest0-21swapper/123:54:431
9334992422,1cyclictest0-21swapper/123:03:251
933499240,0cyclictest0-21swapper/121:19:421
9328992424,0cyclictest0-21swapper/023:22:450
9328992424,0cyclictest0-21swapper/021:51:340
9328992423,1cyclictest62550irq/127-eno123:09:310
9328992423,0cyclictest0-21swapper/022:21:330
932899240,24cyclictest0-21swapper/021:59:170
932899240,23cyclictest20256-21fschecks_count23:02:320
932899240,23cyclictest19355-21latency_hist19:30:010
932899240,23cyclictest10251-21ssh00:27:320
932899240,23cyclictest10239-21sh23:55:130
932899240,23cyclictest0-21swapper/022:40:070
932899240,23cyclictest0-21swapper/021:38:050
932899240,23cyclictest0-21swapper/000:08:150
9340992323,0cyclictest0-21swapper/323:51:403
9340992323,0cyclictest0-21swapper/300:08:483
9340992321,2cyclictest31577-21ssh22:42:053
9340992321,1cyclictest19368-21munin-run19:30:013
9340992320,3cyclictest0-21swapper/322:28:593
9340992320,2cyclictest5763-21sh22:16:303
9340992320,2cyclictest31254-21ssh00:16:083
9340992320,2cyclictest2986-21diskmemload22:23:063
9340992320,2cyclictest15916-21ssh21:54:253
9340992319,4cyclictest0-21swapper/321:23:283
9340992319,3cyclictest2986-21diskmemload22:46:193
9340992319,3cyclictest0-21swapper/323:46:413
934099230,22cyclictest12274-21ssh22:54:073
934099230,20cyclictest2986-21diskmemload23:25:063
934099230,1cyclictest313-21systemd-journal20:05:013
934099230,0cyclictest0-21swapper/323:44:343
9336992323,0cyclictest2986-21diskmemload23:22:052
9336992323,0cyclictest2986-21diskmemload22:57:362
9336992323,0cyclictest2986-21diskmemload22:38:182
9336992323,0cyclictest0-21swapper/223:34:282
9336992323,0cyclictest0-21swapper/223:06:082
9336992323,0cyclictest0-21swapper/200:25:222
9336992321,1cyclictest0-21swapper/222:47:382
9336992321,1cyclictest0-21swapper/220:55:012
9336992320,2cyclictest2986-21diskmemload22:22:392
933699230,2cyclictest25762-21ssh23:38:412
933699230,23cyclictest31180-21ssh23:44:112
933699230,23cyclictest30849-21diskstats23:12:342
933699230,22cyclictest5675-21ssh21:44:152
933699230,22cyclictest4415-21ssh21:11:532
933699230,22cyclictest2986-21diskmemload00:21:162
933699230,22cyclictest11279-21ssh23:56:552
933699230,21cyclictest16140-21ssh00:01:182
933699230,0cyclictest0-21swapper/222:54:192
933699230,0cyclictest0-21swapper/222:25:322
933699230,0cyclictest0-21swapper/221:57:312
933699230,0cyclictest0-21swapper/220:22:312
9334992323,0cyclictest0-21swapper/123:12:111
9334992323,0cyclictest0-21swapper/122:00:341
9334992323,0cyclictest0-21swapper/121:56:081
9334992323,0cyclictest0-21swapper/121:30:471
9334992323,0cyclictest0-21swapper/121:21:031
9334992323,0cyclictest0-21swapper/100:38:101
9334992323,0cyclictest0-21swapper/100:05:111
9334992322,1cyclictest2986-21diskmemload21:28:271
9334992321,1cyclictest0-21swapper/123:42:261
9334992321,1cyclictest0-21swapper/122:37:481
9334992321,1cyclictest0-21swapper/122:22:461
9334992321,1cyclictest0-21swapper/100:01:491
9334992320,2cyclictest24155-21idleruntime-cro19:40:001
9334992320,2cyclictest21550-21cron23:35:011
9334992320,2cyclictest15236-21ssh22:25:121
933499230,2cyclictest6005-21mailstats22:47:401
933499230,23cyclictest0-21swapper/122:15:451
933499230,23cyclictest0-21swapper/100:22:411
933499230,22cyclictest2986-21diskmemload22:51:281
933499230,22cyclictest2910-21sh23:17:191
933499230,22cyclictest0-21swapper/122:40:311
933499230,22cyclictest0-21swapper/122:07:341
933499230,22cyclictest0-21swapper/121:40:481
933499230,1cyclictest3279-21sh21:10:051
933499230,1cyclictest0-21swapper/122:30:231
933499230,0cyclictest0-21swapper/121:46:341
933499230,0cyclictest0-21swapper/120:57:321
9328992323,0cyclictest0-21swapper/023:50:290
9328992323,0cyclictest0-21swapper/022:08:300
9328992323,0cyclictest0-21swapper/000:32:080
9328992322,1cyclictest2986-21diskmemload21:11:390
9328992322,0cyclictest0-21swapper/000:12:230
9328992321,1cyclictest2986-21diskmemload21:40:180
9328992321,1cyclictest24469-21diskmemload21:31:320
9328992321,1cyclictest21001-21sh20:40:000
9328992321,1cyclictest19887-21ssh21:27:180
9328992321,1cyclictest0-21swapper/022:59:540
9328992321,1cyclictest0-21swapper/022:32:330
9328992321,1cyclictest0-21swapper/022:11:030
9328992320,2cyclictest6433-21rm21:45:110
9328992320,2cyclictest606-21dbus-daemon19:20:010
932899231,21cyclictest0-21swapper/022:15:180
932899230,2cyclictest19504-21sh00:03:500
932899230,23cyclictest0-21swapper/000:15:300
932899230,22cyclictest0-21swapper/023:37:220
932899230,22cyclictest0-21swapper/023:12:330
932899230,0cyclictest0-21swapper/023:46:030
932899230,0cyclictest0-21swapper/022:02:310
9340992222,0cyclictest0-21swapper/322:36:043
9340992220,2cyclictest0-21swapper/300:03:233
934099220,2cyclictest20391-21rm22:30:233
934099220,22cyclictest2986-21diskmemload23:21:143
934099220,1cyclictest2986-21diskmemload00:11:483
9336992222,0cyclictest0-21swapper/223:25:102
9336992222,0cyclictest0-21swapper/223:00:282
9336992222,0cyclictest0-21swapper/221:52:082
9336992220,1cyclictest20736-21rm22:30:542
9336992220,1cyclictest0-21swapper/221:45:102
9336992220,0cyclictest0-21swapper/200:10:102
933699220,22cyclictest0-21swapper/223:50:322
933699220,22cyclictest0-21swapper/222:07:242
933699220,1cyclictest8739-21sh22:18:282
933699220,1cyclictest1171-21ssh22:12:122
933699220,0cyclictest2986-21diskmemload21:25:482
933699220,0cyclictest0-21swapper/221:20:252
9334992222,0cyclictest0-21swapper/123:25:541
9334992222,0cyclictest0-21swapper/123:05:021
9334992222,0cyclictest0-21swapper/121:50:551
9334992222,0cyclictest0-21swapper/119:22:311
9334992221,1cyclictest0-21swapper/123:20:251
9334992220,1cyclictest335-21ssh23:46:221
9334992219,2cyclictest23151-21sh23:37:131
933499220,22cyclictest0-21swapper/100:28:201
933499220,21cyclictest2986-21diskmemload22:55:071
933499220,21cyclictest0-21swapper/122:10:301
933499220,21cyclictest0-21swapper/119:47:321
933499220,21cyclictest0-21swapper/100:15:021
933499220,1cyclictest27971-21latency00:12:351
9328992221,1cyclictest0-21swapper/022:50:130
9328992221,1cyclictest0-21swapper/000:38:520
9328992220,1cyclictest3017-21idleruntime-cro21:10:010
9328992220,1cyclictest27006-21ssh23:40:160
932899220,21cyclictest0-21swapper/021:15:040
932899220,21cyclictest0-21swapper/020:17:320
932899220,21cyclictest0-21swapper/019:57:330
932899220,1cyclictest0-21swapper/023:25:420
932899220,1cyclictest0-21swapper/022:45:420
932899220,0cyclictest0-21swapper/022:35:430
932899220,0cyclictest0-21swapper/021:23:180
934099210,21cyclictest0-21swapper/322:00:353
934099210,21cyclictest0-21swapper/321:07:313
933699210,21cyclictest0-21swapper/222:40:202
933699210,21cyclictest0-21swapper/200:30:092
933699210,20cyclictest0-21swapper/223:45:052
933699210,1cyclictest21773-21munin-run19:35:012
933699210,1cyclictest16219-21munin-run20:30:002
933699210,0cyclictest0-21swapper/222:00:052
933699210,0cyclictest0-21swapper/219:17:312
933499210,20cyclictest0-21swapper/119:27:311
933499210,1cyclictest18587-21cron20:35:001
933499210,0cyclictest3755-21apt-get20:05:021
933499210,0cyclictest0-21swapper/121:02:311
9328992120,1cyclictest3744-21idleruntime-cro20:05:010
932899210,21cyclictest18627-21perf20:35:000
932899210,21cyclictest0-21swapper/023:30:150
932899210,21cyclictest0-21swapper/022:26:050
9340992020,0cyclictest31079-21apt-get21:02:313
9340992020,0cyclictest11820-21apt-get20:22:313
9340992020,0cyclictest0-21swapper/319:52:313
9336992020,0cyclictest24624-21apt-get19:42:312
9336992020,0cyclictest0-21swapper/221:02:312
9336992020,0cyclictest0-21swapper/219:20:022
9336992017,2cyclictest8933-21idleruntime-cro20:15:012
933699200,19cyclictest0-21swapper/220:47:312
933699200,0cyclictest28663-21apt-get20:57:322
9334992020,0cyclictest21771-21perf19:35:011
9334992020,0cyclictest0-21swapper/119:42:311
9328992019,0cyclictest0-21swapper/020:27:320
932899200,20cyclictest0-21swapper/019:12:300
932899200,0cyclictest0-21swapper/020:52:320
932899200,0cyclictest0-21swapper/019:20:020
9340991919,0cyclictest9753-21apt-get19:12:303
9340991919,0cyclictest0-21swapper/319:32:313
9340991919,0cyclictest0-21swapper/319:22:313
9340991918,1cyclictest3755-21apt-get20:05:023
9334991919,0cyclictest0-21swapper/120:52:321
9334991919,0cyclictest0-21swapper/120:22:311
9334991919,0cyclictest0-21swapper/120:17:321
932899190,0cyclictest0-21swapper/021:02:310
9340991818,0cyclictest0-21swapper/320:52:323
9340991818,0cyclictest0-21swapper/320:27:323
9340991818,0cyclictest0-21swapper/319:42:313
9336991818,0cyclictest27017-21apt-get19:47:322
9336991818,0cyclictest0-21swapper/221:07:312
9336991818,0cyclictest0-21swapper/220:37:312
9336991818,0cyclictest0-21swapper/220:07:312
9336991818,0cyclictest0-21swapper/219:52:312
9334991818,0cyclictest0-21swapper/120:47:311
9334991818,0cyclictest0-21swapper/120:42:311
9334991818,0cyclictest0-21swapper/119:52:321
9328991815,2cyclictest28962-21turbostat19:50:000
9340991717,0cyclictest22207-21apt-get19:37:313
9340991717,0cyclictest0-21swapper/320:47:313
9340991717,0cyclictest0-21swapper/320:35:003
9336991714,2cyclictest23428-21apt20:45:002
9336991714,2cyclictest19335-21sh19:30:012
9334991717,0cyclictest0-21swapper/119:17:301
9334991715,1cyclictest1050-21apt-get21:07:311
933499170,14cyclictest16192-21cron20:30:001
9328991717,0cyclictest0-21swapper/020:57:320
9328991717,0cyclictest0-21swapper/020:42:310
9328991715,1cyclictest8927-21basename20:15:010
9340991616,0cyclictest0-21swapper/319:57:333
9340991616,0cyclictest0-21swapper/319:47:323
9340991614,1cyclictest19058-21apt-get20:37:313
9334991614,1cyclictest0-21swapper/120:37:421
9328991613,2cyclictest31335-21cron19:55:000
9340991512,2cyclictest12397-21cstates19:17:323
9336991515,0cyclictest0-21swapper/220:32:322
933499150,14cyclictest0-21swapper/120:10:551
934099140,14cyclictest0-21swapper/320:55:303
934099140,14cyclictest0-21swapper/320:15:473
934099140,0cyclictest0-21swapper/320:40:243
9336991414,0cyclictest0-21swapper/220:16:312
933699140,14cyclictest0-21swapper/220:00:162
933699140,14cyclictest0-21swapper/219:55:152
933699140,0cyclictest0-21swapper/219:37:302
933699140,0cyclictest0-21swapper/219:11:502
9334991414,0cyclictest0-21swapper/119:10:481
933499140,14cyclictest0-21swapper/119:55:181
933499140,0cyclictest0-21swapper/120:00:481
932899140,14cyclictest0-21swapper/020:46:410
932899140,14cyclictest0-21swapper/020:05:080
932899140,14cyclictest0-21swapper/019:40:010
932899140,14cyclictest0-21swapper/019:35:140
932899140,0cyclictest0-21swapper/019:30:230
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional