You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-05 - 01:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Sun Jul 05, 2026 00:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3166021210,2sleep30-21swapper/320:33:203
3166021210,2sleep30-21swapper/320:33:203
3166021210,2sleep30-21swapper/320:33:203
1796921030,2sleep3431rcuc/300:13:333
1796921030,2sleep3431rcuc/300:13:333
1796921030,2sleep3431rcuc/300:13:333
145242890,2chrt13018-40mandb00:03:431
145242890,2chrt13018-40mandb00:03:431
319262790,3chrt0-21swapper/120:33:261
319262790,3chrt0-21swapper/120:33:261
319262790,3chrt0-21swapper/120:33:261
272012590,3sleep20-21swapper/200:38:332
272012590,3sleep20-21swapper/200:38:332
151022590,1sleep114880-21/usr/sbin/munin19:48:251
151022590,1sleep114880-21/usr/sbin/munin19:48:251
151022590,1sleep114880-21/usr/sbin/munin19:48:251
46842580,3chrt0-21swapper/320:48:123
161262570,2sleep30-21swapper/300:08:353
161262570,2sleep30-21swapper/300:08:353
30842560,2sleep03087-21grep23:33:350
30842560,2sleep03087-21grep23:33:350
266602560,1sleep3431rcuc/300:38:263
266602560,1sleep3431rcuc/300:38:263
2204995548,4cyclictest0-21swapper/100:03:191
2204995548,4cyclictest0-21swapper/100:03:191
2204995548,4cyclictest0-21swapper/100:03:191
220202540,3sleep30-21swapper/322:58:323
220202540,3sleep30-21swapper/322:58:323
235462530,3sleep30-21swapper/321:38:163
172062510,3sleep30-21swapper/321:18:353
172062510,3sleep30-21swapper/321:18:353
165722470,2sleep3431rcuc/322:43:333
165722470,2sleep3431rcuc/322:43:333
165722470,2sleep3431rcuc/322:43:333
32272430,2sleep13229-21fschecks_count20:43:231
220699391,23cyclictest14555-21taskset00:04:082
220699391,23cyclictest14555-21taskset00:04:082
276252380,2sleep1601-21snmpd20:21:461
276252380,2sleep1601-21snmpd20:21:461
105772380,2sleep110578-21sshd19:33:571
105772380,2sleep110578-21sshd19:33:571
260292370,1sleep10-21swapper/120:18:191
260292370,1sleep10-21swapper/120:18:191
260292370,1sleep10-21swapper/120:18:191
2204993729,6cyclictest8080-21netstat19:28:291
2204993729,6cyclictest8080-21netstat19:28:291
2204993729,6cyclictest8080-21netstat19:28:291
2203993724,9cyclictest13168-21/usr/sbin/munin00:03:290
2203993724,9cyclictest13168-21/usr/sbin/munin00:03:290
61002360,3sleep10-21swapper/120:48:341
275382350,3sleep027543-21unixbench_singl00:38:350
275382350,3sleep027543-21unixbench_singl00:38:350
2203993427,5cyclictest3211-21netstat22:08:290
2203993427,5cyclictest3211-21netstat22:08:290
2203993426,6cyclictest7913-21sendmail22:23:120
2203993426,6cyclictest7913-21sendmail22:23:120
2203993426,6cyclictest7913-21sendmail22:23:120
2204993326,5cyclictest2286-21latency_hist22:08:231
2204993326,5cyclictest2286-21latency_hist22:08:231
2203993326,5cyclictest10821-21fschecks_time21:03:250
2203993326,5cyclictest10821-21fschecks_time21:03:250
220699321,21cyclictest13103-21chrt00:03:192
220699321,21cyclictest13103-21chrt00:03:192
220699321,21cyclictest13103-21chrt00:03:192
2204993226,3cyclictest2807-21sendmail20:43:121
2204993226,3cyclictest2807-21sendmail20:43:121
2204993225,5cyclictest26075-21latency_hist00:38:121
2204993225,5cyclictest26075-21latency_hist00:38:121
2204993224,6cyclictest547-21rs:main1
2204993224,6cyclictest547-21rs:main1
2204993224,6cyclictest22535-21cut20:08:241
2204993224,6cyclictest22535-21cut20:08:241
2204993223,7cyclictest10349-21latency_hist21:03:111
2203993224,6cyclictest7497-21netstat20:53:300
2203993224,6cyclictest7497-21netstat20:53:300
2203993224,6cyclictest17749-21fschecks_time22:48:240
2203993224,6cyclictest17749-21fschecks_time22:48:240
2203993224,6cyclictest11806-21needreboot19:38:290
2203993224,6cyclictest11806-21needreboot19:38:290
199912320,2sleep1281ktimers/122:53:281
199912320,2sleep1281ktimers/122:53:281
285932310,5sleep028596-21unixbench-2d21:48:350
285932310,5sleep028596-21unixbench-2d21:48:350
2204993125,4cyclictest9804-21latency_hist23:53:251
2204993124,5cyclictest22810-21cut00:28:251
2203993127,3cyclictest21036-21cron22:58:110
2203993127,3cyclictest21036-21cron22:58:110
2204993023,5cyclictest8855-21netstat22:23:281
2204993023,5cyclictest8855-21netstat22:23:281
2204993023,5cyclictest8855-21netstat22:23:281
2204993023,5cyclictest32703-21kworker/u9:2-writeback21:28:231
2204993023,5cyclictest32703-21kworker/u9:2-writeback21:28:231
2204993022,6cyclictest2200-21cyclictest21:54:211
2204993022,6cyclictest2200-21cyclictest21:54:211
2204993022,6cyclictest2200-21cyclictest19:58:061
2204993022,6cyclictest2200-21cyclictest19:58:061
2203993024,4cyclictest2200-21cyclictest23:03:130
2203993024,4cyclictest2200-21cyclictest23:03:130
2203993023,5cyclictest32286-21timerwakeupswit21:58:340
2203993023,5cyclictest32286-21timerwakeupswit21:58:340
2203993023,5cyclictest32286-21timerwakeupswit21:58:340
2203993022,7cyclictest572-21cups-browsed21:33:330
220999292,16cyclictest0-21swapper/300:03:273
220999292,16cyclictest0-21swapper/300:03:273
220999292,16cyclictest0-21swapper/300:03:273
2204992923,4cyclictest29460-21fschecks_count21:53:231
2204992923,4cyclictest29460-21fschecks_count21:53:231
2204992923,4cyclictest22463-21unixbench_multi22:58:361
2204992923,4cyclictest22463-21unixbench_multi22:58:361
2204992922,5cyclictest4286-21kernelversion23:38:291
2204992922,5cyclictest4286-21kernelversion23:38:291
2204992921,6cyclictest27877-21threads23:13:311
2204992921,6cyclictest27877-21threads23:13:311
2204992920,7cyclictest22146-21head21:33:261
2203992921,6cyclictest773-21apache200:23:010
2203992921,6cyclictest773-21apache200:23:010
2203992920,7cyclictest13909-21wc19:43:320
2203992920,7cyclictest13909-21wc19:43:320
2203992920,7cyclictest13909-21wc19:43:320
81592280,3sleep20-21swapper/223:48:282
81592280,3sleep20-21swapper/223:48:282
2204992823,4cyclictest749-21runrttasks20:28:291
2204992823,4cyclictest749-21runrttasks20:28:291
2204992823,4cyclictest749-21runrttasks20:28:291
2204992823,2cyclictest0-21swapper/122:13:311
2204992823,2cyclictest0-21swapper/122:13:311
2203992823,4cyclictest749-21runrttasks23:23:310
2203992823,4cyclictest749-21runrttasks23:23:310
2203992823,4cyclictest749-21runrttasks23:23:310
2203992822,5cyclictest26352-21processes21:43:320
2203992822,5cyclictest26352-21processes21:43:320
2203992821,5cyclictest9232-21latency_hist23:53:230
2203992821,5cyclictest9232-21latency_hist23:53:230
2204992722,2cyclictest0-21swapper/123:13:221
2204992722,2cyclictest0-21swapper/123:13:221
2204992722,2cyclictest0-21swapper/123:13:221
2204992721,4cyclictest0-21swapper/121:03:341
2204992721,4cyclictest0-21swapper/121:03:341
2204992720,5cyclictest30844-21cut23:23:241
2204992720,5cyclictest30844-21cut23:23:241
2204992720,5cyclictest16168-21kernelversion22:43:271
2204992720,5cyclictest16168-21kernelversion22:43:271
2204992720,5cyclictest16168-21kernelversion22:43:271
2204992720,4cyclictest0-21swapper/100:23:331
2204992719,5cyclictest15825-21ntp_kernel_pll_00:08:311
2204992719,5cyclictest15825-21ntp_kernel_pll_00:08:311
2203992722,3cyclictest2200-21cyclictest23:05:340
2203992722,3cyclictest2200-21cyclictest23:05:340
2203992722,3cyclictest2200-21cyclictest23:05:340
2203992722,3cyclictest0-21swapper/023:38:330
2203992722,3cyclictest0-21swapper/023:38:330
2203992721,5cyclictest21469-21switchtime20:03:350
2203992721,5cyclictest21469-21switchtime20:03:350
2203992721,5cyclictest21469-21switchtime20:03:350
2203992720,5cyclictest550-21cron19:33:240
2203992720,5cyclictest550-21cron19:33:240
2203992719,6cyclictest2200-21cyclictest22:13:310
2203992719,6cyclictest2200-21cyclictest22:13:310
2206992620,4cyclictest53012sleep222:13:332
2206992620,4cyclictest53012sleep222:13:332
2204992622,3cyclictest28347-21gltestperf20:23:251
2204992622,3cyclictest28347-21gltestperf20:23:251
2204992621,3cyclictest6218-21kworker/u9:3-xprtiod21:26:451
2204992620,3cyclictest0-21swapper/100:18:251
2204992620,3cyclictest0-21swapper/100:18:251
2204992618,6cyclictest18054-21grep21:23:231
2204992618,6cyclictest18054-21grep21:23:231
2204992616,8cyclictest24323-21latency_hist00:33:171
2204992616,8cyclictest24323-21latency_hist00:33:171
2203992622,2cyclictest0-21swapper/022:33:350
2203992622,2cyclictest0-21swapper/022:33:350
2203992621,4cyclictest26762-21runrttasks20:18:280
2203992621,4cyclictest26762-21runrttasks20:18:280
2203992621,3cyclictest0-21swapper/019:28:120
2203992621,3cyclictest0-21swapper/019:28:120
2203992620,5cyclictest12918-21kernelversion21:08:260
2203992620,5cyclictest12918-21kernelversion21:08:260
2203992620,5cyclictest12918-21kernelversion21:08:260
2203992620,4cyclictest2200-21cyclictest23:58:330
2203992620,4cyclictest2200-21cyclictest23:58:330
2203992620,4cyclictest2200-21cyclictest23:58:330
2203992618,6cyclictest30269-21cat21:53:330
2203992618,6cyclictest30269-21cat21:53:330
50872250,3sleep30-21swapper/322:13:303
50872250,3sleep30-21swapper/322:13:303
2204992520,3cyclictest32703-21kworker/u9:2-rpciod21:29:151
2204992520,3cyclictest32703-21kworker/u9:2-rpciod21:29:151
2204992520,3cyclictest26343-21kworker/u9:3-writeback00:18:241
2204992520,3cyclictest26343-21kworker/u9:3-writeback00:18:241
2204992520,3cyclictest14580-21fschecks_time21:13:261
2204992520,3cyclictest14580-21fschecks_time21:13:261
2204992520,3cyclictest14580-21fschecks_time21:13:261
2204992519,3cyclictest0-21swapper/121:43:381
2204992519,3cyclictest0-21swapper/121:43:381
2204992517,6cyclictest8313-21fschecks_count22:23:231
2204992517,6cyclictest8313-21fschecks_count22:23:231
2204992512,10cyclictest22147-21cron20:08:111
2204992512,10cyclictest22147-21cron20:08:111
2204992512,10cyclictest22147-21cron20:08:111
2203992521,2cyclictest0-21swapper/019:58:260
2203992521,2cyclictest0-21swapper/019:58:260
2203992520,3cyclictest0-21swapper/020:33:270
2203992520,3cyclictest0-21swapper/020:33:270
2203992520,3cyclictest0-21swapper/020:33:270
2203992520,3cyclictest0-21swapper/019:53:120
2203992520,3cyclictest0-21swapper/019:53:120
2203992520,3cyclictest0-21swapper/019:53:120
2203992520,3cyclictest0-21swapper/000:23:290
2203992520,3cyclictest0-21swapper/000:23:290
2203992520,2cyclictest0-21swapper/023:23:240
2203992518,5cyclictest23248-21sendmail_mailst20:08:330
2203992518,5cyclictest23248-21sendmail_mailst20:08:330
2203992513,10cyclictest14678-21systemd-journal20:28:230
2203992513,10cyclictest14678-21systemd-journal20:28:230
2203992513,10cyclictest14678-21systemd-journal20:28:230
277912240,3sleep20-21swapper/223:13:312
277912240,3sleep20-21swapper/223:13:312
2204992418,3cyclictest0-21swapper/123:03:311
2204992418,3cyclictest0-21swapper/123:03:311
2204992418,3cyclictest0-21swapper/123:03:311
2204992416,6cyclictest5825-21sshd23:43:241
2204992416,6cyclictest5825-21sshd23:43:241
2204992416,6cyclictest5825-21sshd23:43:241
2203992420,2cyclictest0-21swapper/021:23:120
2203992420,2cyclictest0-21swapper/021:23:120
2203992419,4cyclictest15354-21timerandwakeup21:13:350
2203992419,4cyclictest15354-21timerandwakeup21:13:350
2203992419,4cyclictest15354-21timerandwakeup21:13:350
2203992419,3cyclictest7652-21latency_hist23:48:240
2203992419,3cyclictest7652-21latency_hist23:48:240
2203992419,3cyclictest7652-21latency_hist23:48:240
2203992419,3cyclictest0-21swapper/023:13:180
2203992419,3cyclictest0-21swapper/023:13:180
2203992419,3cyclictest0-21swapper/023:13:180
2203992419,3cyclictest0-21swapper/019:53:240
2203992419,3cyclictest0-21swapper/019:53:240
2203992417,5cyclictest17416-21latency_hist22:48:180
2203992417,5cyclictest11186-21latency_hist23:58:170
2203992417,5cyclictest11186-21latency_hist23:58:170
2203992417,5cyclictest11186-21latency_hist23:58:170
2203992417,4cyclictest0-21swapper/020:38:330
2203992417,4cyclictest0-21swapper/020:38:330
2203992417,4cyclictest0-21swapper/020:38:330
2203992416,6cyclictest894-21fschecks_time22:03:250
2203992416,6cyclictest894-21fschecks_time22:03:250
2203992416,6cyclictest894-21fschecks_time22:03:250
2204992319,2cyclictest0-21swapper/123:33:261
2204992319,2cyclictest0-21swapper/123:33:261
2204992319,2cyclictest0-21swapper/121:10:571
2204992319,2cyclictest0-21swapper/121:10:571
2204992319,2cyclictest0-21swapper/121:10:571
2204992318,3cyclictest0-21swapper/122:53:241
2204992318,3cyclictest0-21swapper/122:53:241
2204992318,3cyclictest0-21swapper/120:43:271
2204992317,3cyclictest0-21swapper/121:38:271
2204992317,3cyclictest0-21swapper/121:38:271
2204992316,5cyclictest550-21cron22:38:221
2204992316,5cyclictest550-21cron22:38:221
2204992316,5cyclictest550-21cron22:38:221
2204992314,7cyclictest26828-21ls00:38:271
2204992314,7cyclictest26828-21ls00:38:271
2203992317,5cyclictest23797-21unixbench_singl00:28:350
2203992317,5cyclictest23797-21unixbench_singl00:28:350
2203992317,4cyclictest0-21swapper/022:38:330
2203992317,4cyclictest0-21swapper/022:38:330
2203992317,4cyclictest0-21swapper/022:38:330
2203992316,5cyclictest5075-21fschecks_count20:48:240
2203992315,6cyclictest15616-21cut00:08:280
2203992315,6cyclictest15616-21cut00:08:280
2203992314,7cyclictest29398-21head23:18:300
2203992313,8cyclictest18393-21sshd00:15:380
2203992313,8cyclictest18393-21sshd00:15:380
2203992313,8cyclictest18393-21sshd00:15:380
2204992217,3cyclictest5801-21gltestperf19:23:251
2204992217,3cyclictest5801-21gltestperf19:23:251
2204992217,3cyclictest0-21swapper/123:33:031
2204992217,3cyclictest0-21swapper/119:43:351
2204992217,3cyclictest0-21swapper/119:43:351
2204992217,3cyclictest0-21swapper/119:43:351
2204992216,5cyclictest7517-21/usr/sbin/munin23:48:321
2204992215,4cyclictest0-21swapper/122:08:321
2204992215,4cyclictest0-21swapper/122:08:321
2204992214,6cyclictest10750-21ntp_kernel_err22:28:291
2204992214,6cyclictest10750-21ntp_kernel_err22:28:291
2204992212,8cyclictest14678-21systemd-journal20:56:141
2204992212,8cyclictest14678-21systemd-journal20:56:141
2203992218,3cyclictest28777-21cpuspeed23:18:230
2203992218,3cyclictest28777-21cpuspeed23:18:230
2203992217,3cyclictest0-21swapper/022:48:500
2203992217,3cyclictest0-21swapper/022:48:500
2203992217,3cyclictest0-21swapper/021:28:320
2203992217,3cyclictest0-21swapper/021:28:320
2203992216,3cyclictest0-21swapper/022:28:320
2203992216,3cyclictest0-21swapper/022:28:320
2203992213,7cyclictest10119-21latency_hist22:28:230
2203992213,7cyclictest10119-21latency_hist22:28:230
2203992213,7cyclictest10119-21latency_hist22:28:230
2203992212,8cyclictest14678-21systemd-journal20:58:440
2204992117,2cyclictest0-21swapper/122:33:271
2204992117,2cyclictest0-21swapper/122:33:271
2204992116,3cyclictest90-1pr/ttyS022:18:271
2204992116,3cyclictest11478-21cut19:38:251
2204992116,3cyclictest11478-21cut19:38:251
2204992116,3cyclictest0-21swapper/121:58:331
2204992116,3cyclictest0-21swapper/121:58:331
2204992116,3cyclictest0-21swapper/121:58:331
2204992115,4cyclictest0-21swapper/122:03:341
2204992115,3cyclictest0-21swapper/123:58:221
2204992115,3cyclictest0-21swapper/123:58:221
2204992115,3cyclictest0-21swapper/123:58:221
2204992115,3cyclictest0-21swapper/123:23:261
2204992115,3cyclictest0-21swapper/123:23:261
2204992115,3cyclictest0-21swapper/123:23:261
2203992117,2cyclictest0-21swapper/019:28:280
2203992117,2cyclictest0-21swapper/019:28:280
2203992117,2cyclictest0-21swapper/019:28:280
2203992117,2cyclictest0-21swapper/000:33:340
2203992117,2cyclictest0-21swapper/000:33:340
2203992116,4cyclictest3543-21kernelversion20:43:260
2203992116,3cyclictest0-21swapper/023:28:340
2203992116,3cyclictest0-21swapper/021:38:250
2203992116,3cyclictest0-21swapper/021:38:250
2203992115,4cyclictest0-21swapper/021:23:330
2203992115,4cyclictest0-21swapper/021:23:330
2203992115,4cyclictest0-21swapper/021:23:330
2204992015,3cyclictest0-21swapper/100:13:251
220399208,9cyclictest28328-21sed20:23:250
220399208,9cyclictest28328-21sed20:23:250
2203992015,3cyclictest0-21swapper/020:13:350
2203992015,3cyclictest0-21swapper/020:13:350
2203992015,3cyclictest0-21swapper/020:13:350
220999184,11cyclictest0-21swapper/300:04:083
220999184,11cyclictest0-21swapper/300:04:083
220999181,11cyclictest20477-21chrt22:53:343
220999181,11cyclictest20477-21chrt22:53:343
220699183,9cyclictest0-21swapper/221:48:562
220699183,9cyclictest0-21swapper/221:48:562
220699183,9cyclictest0-21swapper/219:44:562
220699183,9cyclictest0-21swapper/219:44:562
220699183,9cyclictest0-21swapper/219:44:562
220699183,10cyclictest0-21swapper/219:29:572
220699183,10cyclictest0-21swapper/219:29:572
220699183,10cyclictest0-21swapper/219:29:572
220699181,13cyclictest5777-21chrt19:23:252
220699181,13cyclictest5777-21chrt19:23:252
220699181,11cyclictest10713-21taskset23:53:342
220699181,11cyclictest10713-21taskset23:53:342
220699181,11cyclictest10713-21taskset23:53:342
220999173,10cyclictest26034-21chrt21:43:273
220999173,10cyclictest26034-21chrt21:43:273
220999172,10cyclictest0-21swapper/323:03:273
220999172,10cyclictest0-21swapper/323:03:273
220999172,10cyclictest0-21swapper/323:03:273
220999171,11cyclictest27298-21taskset23:13:243
220999171,11cyclictest27298-21taskset23:13:243
220699173,9cyclictest0-21swapper/221:23:562
220699173,9cyclictest0-21swapper/221:23:562
220699173,9cyclictest0-21swapper/221:23:562
220699173,9cyclictest0-21swapper/220:54:572
220699173,9cyclictest0-21swapper/220:54:572
220699171,12cyclictest6083-21chrt23:43:252
220699171,12cyclictest6083-21chrt23:43:252
220699171,12cyclictest6083-21chrt23:43:252
220699171,12cyclictest24053-21chrt23:03:342
220699171,12cyclictest24053-21chrt23:03:342
220699171,12cyclictest24053-21chrt23:03:342
220699171,12cyclictest10357-21chrt22:28:242
220699171,12cyclictest10357-21chrt22:28:242
220699171,11cyclictest23118-21chrt20:08:312
220699171,11cyclictest23118-21chrt20:08:312
220699171,10cyclictest16410-21chrt21:18:262
220699171,10cyclictest16410-21chrt21:18:262
152832170,2sleep20-21swapper/219:48:252
152832170,2sleep20-21swapper/219:48:252
152832170,2sleep20-21swapper/219:48:252
220999169,5cyclictest131662sleep321:08:313
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional