You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-17 - 10:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot5s.osadl.org (updated Fri Apr 17, 2026 00:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
197850995043,4cyclictest172325-21kworker/u8:3+events_unbound19:21:513
197837994937,9cyclictest121ktimers/023:20:130
197846994841,4cyclictest291124-21munin-run23:40:002
197841994839,5cyclictest201782-21munin-plugin-st19:20:011
197841994819,25cyclictest241015-21grep21:10:251
197841994635,8cyclictest231ktimers/123:10:121
197841994635,8cyclictest231ktimers/123:10:111
197841994635,7cyclictest251703-21idleruntime-cro21:45:001
197841994613,29cyclictest0-21swapper/122:15:001
197841994613,29cyclictest0-21swapper/122:15:001
197837994638,5cyclictest274076-21idleruntime-cro22:50:000
197850994539,4cyclictest227699-21kworker/u8:2+events_unbound21:07:093
19784199441,13cyclictest223939-21tail20:20:261
197850994334,6cyclictest312029-21sendmail00:40:003
197841994333,7cyclictest247858-21unixbench_singl21:30:271
19784199432,12cyclictest277460-21cron23:00:001
197837994331,8cyclictest121ktimers/021:05:140
197841994221,17cyclictest0-21swapper/120:10:221
197841994218,21cyclictest0-21swapper/123:55:261
197841994214,3cyclictest202022-21/usr/sbin/munin19:20:251
197841994212,7cyclictest304296-21grep00:15:261
19784199421,13cyclictest273596-21tail22:45:251
197837994227,11cyclictest121ktimers/022:40:110
197846994128,8cyclictest244619-21sshd21:22:102
197841994113,4cyclictest274523-21fschecks_time22:50:171
197837994129,10cyclictest121ktimers/000:05:120
197841994016,20cyclictest0-21swapper/123:50:011
197841994015,22cyclictest0-21swapper/123:20:261
197841994011,3cyclictest0-21swapper/120:40:001
197850993931,6cyclictest391ktimers/320:30:133
197850993931,4cyclictest282586-21sh23:15:003
197850993931,4cyclictest282586-21sh23:15:003
197846993928,8cyclictest215636-21sendmail20:00:002
197841993927,8cyclictest243163-21sendmail21:20:001
197841993913,3cyclictest289647-21/usr/sbin/munin23:35:181
197850993830,6cyclictest391ktimers/320:45:113
19784199382,11cyclictest227779-21cron20:35:011
197841993814,6cyclictest258507-21sh22:05:001
197841993813,3cyclictest265775-21/usr/sbin/munin22:25:151
197841993812,3cyclictest206900-21latency_hist19:35:011
197850993728,4cyclictest0-21swapper/322:40:243
19784699377,3cyclictest629-21snmpd21:50:122
197841993726,8cyclictest310299-21sh00:35:001
19784199372,11cyclictest312000-21cron00:40:001
197850993627,6cyclictest294119-21grep23:45:243
197850993626,6cyclictest208327-21grep19:35:243
197850993621,6cyclictest280805-21kworker/u8:3+events_unbound23:26:123
197846993629,3cyclictest0-21swapper/200:00:132
197846993626,8cyclictest206895-21auditctl19:35:002
19784199361,11cyclictest212229-21sh19:50:001
197837993627,5cyclictest263847-21sendmail22:20:010
197850993525,7cyclictest391ktimers/322:00:123
197841993523,9cyclictest216895-21grep20:00:251
197841993512,4cyclictest300073-21fschecks_time00:05:151
197841993512,3cyclictest0-21swapper/123:55:011
19783799352,26cyclictest0-21swapper/021:43:140
197850993426,5cyclictest217341-21cron20:05:003
197846993426,4cyclictest258505-21cron22:05:002
197846993426,4cyclictest0-21swapper/219:35:152
197846993425,6cyclictest270231-21grep22:35:252
197846993425,5cyclictest0-21swapper/222:50:162
197846993418,13cyclictest32-21ksoftirqd/220:00:122
197841993426,4cyclictest0-21swapper/121:05:011
197837993426,4cyclictest0-21swapper/021:00:220
197850993326,5cyclictest253610-21kworker/u8:0+events_unbound22:27:213
197850993325,4cyclictest0-21swapper/323:15:153
197850993320,9cyclictest0-21swapper/300:20:003
197850993317,9cyclictest267071-21kworker/u8:1+events_unbound22:35:113
197846993326,3cyclictest0-21swapper/200:30:002
19784199332,9cyclictest164-21systemd-journal21:37:121
197841993324,6cyclictest246557-21cron21:30:001
19783799334,10cyclictest231205-21cron20:45:000
197837993326,4cyclictest0-21swapper/021:40:000
197837993322,8cyclictest246567-21idleruntime-cro21:30:000
197850993225,5cyclictest179047-21kworker/u8:0+events_unbound19:26:423
197850993223,6cyclictest251686-21auditctl21:45:003
197846993224,4cyclictest0-21swapper/219:45:262
197846993218,10cyclictest311ktimers/219:40:122
197841993225,4cyclictest270649-21kthreadd22:40:001
197837993222,6cyclictest0-21swapper/000:10:260
197837993220,5cyclictest131rcu_preempt19:59:020
197850993123,3cyclictest0-21swapper/320:20:153
197850993122,6cyclictest0-21swapper/320:30:013
197850993120,7cyclictest0-21swapper/319:45:013
197850993118,4cyclictest288496-21/usr/sbin/munin23:30:193
197850993118,4cyclictest288496-21/usr/sbin/munin23:30:193
197846993123,4cyclictest0-21swapper/223:25:262
197841993124,4cyclictest0-21swapper/119:40:251
19783799319,5cyclictest151rcuc/000:00:010
19783799315,14cyclictest11-21ksoftirqd/000:30:250
19783799314,7cyclictest287845-21latency_hist23:30:000
197837993123,4cyclictest0-21swapper/020:10:000
197837993123,4cyclictest0-21swapper/020:10:000
197850993023,5cyclictest258470-21kworker/u8:2+events_unbound22:06:083
197850993022,4cyclictest0-21swapper/323:20:263
197850993022,4cyclictest0-21swapper/319:10:243
197850993022,4cyclictest0-21swapper/300:30:003
197850993022,4cyclictest0-21swapper/300:05:253
197846993024,4cyclictest253610-21kworker/u8:0+events_unbound23:20:432
197846993024,4cyclictest253610-21kworker/u8:0+events_unbound22:15:012
197846993023,4cyclictest304396-21kworker/u8:1+events_unbound00:30:042
197846993022,3cyclictest0-21swapper/220:50:012
197846993022,3cyclictest0-21swapper/219:25:002
197846993021,6cyclictest261991-21ls22:15:002
197846993021,6cyclictest261991-21ls22:15:002
197846993020,6cyclictest0-21swapper/221:40:002
197841993021,5cyclictest287719-21sh23:30:001
197837993023,4cyclictest121ktimers/023:30:240
197837993023,4cyclictest121ktimers/023:30:240
197837993023,4cyclictest0-21swapper/023:10:250
197837993023,4cyclictest0-21swapper/023:10:250
197837993021,6cyclictest226189-21cat20:30:010
197837993021,6cyclictest0-21swapper/021:10:240
197837993019,7cyclictest207096-21sed19:35:010
197837993011,4cyclictest0-21swapper/023:15:230
19785099298,5cyclictest131rcu_preempt22:30:263
19785099293,4cyclictest131rcu_preempt22:45:113
19785099291,14cyclictest0-21swapper/322:50:133
197846992922,4cyclictest0-21swapper/221:10:242
197846992922,4cyclictest0-21swapper/220:15:242
197846992921,6cyclictest288943-21unixbench_multi23:30:232
197846992921,6cyclictest288943-21unixbench_multi23:30:232
197846992921,4cyclictest0-21swapper/200:10:002
197841992918,7cyclictest1-21systemd00:02:401
197841992917,8cyclictest302641-21grep00:10:261
19783799299,4cyclictest151rcuc/022:10:010
19783799298,6cyclictest11-21ksoftirqd/022:50:260
19783799298,5cyclictest11-21ksoftirqd/019:30:010
19783799297,5cyclictest11-21ksoftirqd/019:15:190
19783799295,7cyclictest11-21ksoftirqd/000:25:140
19783799295,6cyclictest11-21ksoftirqd/021:30:260
19783799295,3cyclictest151rcuc/019:20:240
197837992922,4cyclictest236305-21cron21:00:000
197837992922,3cyclictest0-21swapper/023:00:000
197837992918,4cyclictest131rcu_preempt20:34:170
19785099289,4cyclictest131rcu_preempt00:00:093
19785099282,4cyclictest131rcu_preempt21:15:123
197850992821,3cyclictest0-21swapper/323:40:273
197850992820,6cyclictest179047-21kworker/u8:0+events_unbound19:30:433
197850992819,6cyclictest246188-21kworker/u8:2+events_unbound21:35:193
197846992822,3cyclictest0-21swapper/223:50:292
197846992820,4cyclictest0-21swapper/219:30:002
197846992818,7cyclictest256802-21sendmail22:00:002
197841992820,5cyclictest0-21swapper/122:05:251
19783799289,6cyclictest11-21ksoftirqd/019:53:500
19783799287,5cyclictest11-21ksoftirqd/020:45:220
19783799286,5cyclictest0-21swapper/022:30:160
19783799284,5cyclictest151rcuc/020:15:190
197837992818,6cyclictest0-21swapper/023:50:280
197837992817,8cyclictest199775-21sed19:15:010
197837992815,7cyclictest131rcu_preempt21:18:250
197837992813,4cyclictest131rcu_preempt23:00:200
197837992811,10cyclictest131rcu_preempt22:35:210
197837992810,10cyclictest131rcu_preempt23:43:120
197850992720,5cyclictest204816-21kworker/u8:1+events_unbound19:48:483
197850992720,4cyclictest0-21swapper/320:20:013
19785099271,9cyclictest0-21swapper/300:10:133
197850992718,5cyclictest229523-21sendmail-msp20:40:003
19785099271,7cyclictest0-21swapper/321:50:133
19785099271,13cyclictest0-21swapper/320:53:083
19785099271,13cyclictest0-21swapper/320:53:083
197850992710,5cyclictest131rcu_preempt22:20:123
197846992720,5cyclictest145-21jbd2/mmcblk1p2-821:00:422
197846992720,3cyclictest0-21swapper/200:00:002
197846992719,4cyclictest0-21swapper/223:40:272
197846992719,4cyclictest0-21swapper/220:40:262
197841992719,5cyclictest0-21swapper/119:50:251
197841992718,5cyclictest0-21swapper/123:15:151
19783799277,6cyclictest0-21swapper/000:00:140
19783799276,4cyclictest11-21ksoftirqd/022:20:160
19783799275,6cyclictest0-21swapper/023:46:000
19783799275,4cyclictest151rcuc/021:45:250
19783799275,4cyclictest11-21ksoftirqd/020:35:200
197837992718,7cyclictest121ktimers/019:40:160
197837992717,6cyclictest304290-21grep00:15:260
197837992713,4cyclictest131rcu_preempt21:56:530
197850992620,4cyclictest258470-21kworker/u8:2+events_unbound22:55:403
197850992620,3cyclictest0-21swapper/320:05:243
197850992620,3cyclictest0-21swapper/320:05:243
197850992619,4cyclictest0-21swapper/321:00:223
197850992618,5cyclictest0-21swapper/323:00:253
197850992618,4cyclictest0-21swapper/300:30:263
19785099261,7cyclictest248009-21taskset21:32:023
197850992613,4cyclictest131rcu_preempt00:23:243
19785099261,13cyclictest0-21swapper/322:10:513
19785099261,13cyclictest0-21swapper/322:10:503
19785099261,11cyclictest0-21swapper/321:27:103
19784699266,5cyclictest131rcu_preempt22:40:112
197846992620,3cyclictest0-21swapper/220:55:242
197846992620,3cyclictest0-21swapper/200:10:182
197846992619,5cyclictest145-21jbd2/mmcblk1p2-819:15:422
197846992619,3cyclictest0-21swapper/220:30:262
197846992617,6cyclictest0-21swapper/200:35:242
197846992616,8cyclictest304868-21sed00:20:022
197841992619,3cyclictest0-21swapper/121:45:201
197841992615,7cyclictest1-21systemd19:26:511
19783799269,4cyclictest131rcu_preempt22:02:160
19783799266,6cyclictest0-21swapper/000:20:230
19783799266,5cyclictest11-21ksoftirqd/022:10:110
19783799266,5cyclictest11-21ksoftirqd/022:10:110
19783799266,17cyclictest121ktimers/021:51:240
19783799266,11cyclictest0-21swapper/020:54:520
19783799266,11cyclictest0-21swapper/020:54:520
19783799262,7cyclictest0-21swapper/019:38:570
197837992612,4cyclictest131rcu_preempt20:10:450
197850992519,4cyclictest267071-21kworker/u8:1+events_unbound23:07:383
197850992519,4cyclictest267071-21kworker/u8:1+events_unbound23:07:373
197850992519,4cyclictest258470-21kworker/u8:2+events_unbound22:15:153
197850992519,4cyclictest172325-21kworker/u8:3+events_unbound19:15:043
197850992518,5cyclictest241053-21kworker/u8:0+events_unbound21:22:413
197850992518,5cyclictest241053-21kworker/u8:0+events_unbound21:12:133
197850992518,5cyclictest227699-21kworker/u8:2+events_unbound20:58:543
19785099251,13cyclictest0-21swapper/323:53:583
197846992520,3cyclictest260395-21latency_hist22:10:012
197846992519,4cyclictest221641-21kworker/u8:1+events_unbound20:27:002
197846992519,3cyclictest0-21swapper/223:00:002
197846992518,3cyclictest0-21swapper/200:20:232
197846992516,6cyclictest0-21swapper/223:20:002
197846992514,8cyclictest247548-21mailstats21:30:252
197841992519,3cyclictest0-21swapper/123:05:161
197841992519,3cyclictest0-21swapper/123:05:161
197841992518,4cyclictest306226-21grep00:20:251
197841992516,6cyclictest262333-21gmain22:15:021
197841992516,6cyclictest231ktimers/119:35:091
19783799253,5cyclictest11-21ksoftirqd/000:35:130
197837992519,3cyclictest11-21ksoftirqd/023:40:000
197837992518,5cyclictest121ktimers/022:25:260
197837992518,3cyclictest243555-21fschecks_count21:20:170
197850992419,3cyclictest0-21swapper/321:55:233
197850992418,3cyclictest253157-21close21:47:143
197850992410,8cyclictest40-21ksoftirqd/323:36:303
197846992418,3cyclictest0-21swapper/220:05:162
197846992418,3cyclictest0-21swapper/220:05:162
197846992416,4cyclictest253391-21latency_hist21:50:002
197846992415,6cyclictest264285-21fschecks_time22:20:172
197841992416,4cyclictest0-21swapper/120:50:251
197841992416,4cyclictest0-21swapper/120:50:251
197841992415,6cyclictest268403-21ls22:30:251
197841992415,5cyclictest254627-21grep21:50:261
19783799244,7cyclictest11-21ksoftirqd/019:45:220
19783799244,6cyclictest11-21ksoftirqd/023:05:200
19783799244,6cyclictest11-21ksoftirqd/023:05:200
197837992417,4cyclictest0-21swapper/020:20:260
197837992414,7cyclictest0-21swapper/020:00:200
19785099235,10cyclictest0-21swapper/319:55:163
19785099231,12cyclictest0-21swapper/320:10:053
197846992317,3cyclictest0-21swapper/222:45:182
197846992317,3cyclictest0-21swapper/221:40:252
197846992316,4cyclictest311ktimers/221:27:562
197846992315,3cyclictest282134-21unixbench_multi23:10:252
197846992315,3cyclictest282134-21unixbench_multi23:10:252
197846992314,6cyclictest0-21swapper/221:10:002
197841992317,3cyclictest0-21swapper/123:40:241
197841992317,3cyclictest0-21swapper/120:25:141
197841992316,3cyclictest0-21swapper/122:20:251
197841992315,5cyclictest0-21swapper/121:20:271
197841992314,6cyclictest217958-21grep20:05:171
197841992314,6cyclictest217958-21grep20:05:171
197841992312,8cyclictest230731-21grep20:40:251
197850992216,4cyclictest289192-21kworker/u8:1+events_unbound23:55:053
197850992215,4cyclictest212989-21ntpq19:50:203
197846992216,4cyclictest145-21jbd2/mmcblk1p2-822:30:222
197846992215,4cyclictest311ktimers/223:00:162
197841992216,3cyclictest0-21swapper/119:10:141
197846992116,3cyclictest0-21swapper/220:10:092
197846992113,5cyclictest0-21swapper/221:15:112
197846992113,4cyclictest0-21swapper/220:35:232
197841992115,3cyclictest0-21swapper/121:55:171
197841992114,3cyclictest0-21swapper/100:25:191
197841992113,4cyclictest277878-21fschecks_count23:00:151
197841992112,6cyclictest0-21swapper/121:00:001
197846992014,3cyclictest0-21swapper/219:50:152
197846992013,5cyclictest311ktimers/223:10:002
197846992013,5cyclictest311ktimers/223:10:002
197846992012,6cyclictest311ktimers/222:25:232
197841992014,3cyclictest0-21swapper/123:30:131
197841992014,3cyclictest0-21swapper/123:30:131
197841992014,3cyclictest0-21swapper/121:10:011
197846991912,3cyclictest0-21swapper/220:55:012
197846991912,3cyclictest0-21swapper/220:55:012
197846991911,5cyclictest223202-21/usr/sbin/munin20:20:172
197841991913,3cyclictest0-21swapper/120:15:191
19784199191,13cyclictest232068-21ps20:45:201
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional