You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-05 - 13:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot5s.osadl.org (updated Fri Dec 05, 2025 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
327934995629,22cyclictest405860-21idleruntime-cro22:55:010
327934995432,20cyclictest121ktimers/023:05:150
327946995243,5cyclictest386877-21sendmail22:00:003
327946995142,6cyclictest346138-21fschecks_count20:00:183
327946995142,6cyclictest346138-21fschecks_count20:00:173
327939995140,7cyclictest421196-21grep23:35:251
327934994840,5cyclictest359364-21sh20:40:010
327939994734,8cyclictest436553-21grep00:20:271
327939994734,8cyclictest436553-21grep00:20:271
327943994539,4cyclictest362795-21munin-plugin-st20:50:012
327939994534,8cyclictest231ktimers/100:25:151
327946994332,9cyclictest391ktimers/321:25:153
327946994332,7cyclictest360675-21unixbench_singl20:40:293
327943994315,25cyclictest1-21systemd23:02:572
327943994313,6cyclictest400864-21sed22:40:002
327946994234,6cyclictest391ktimers/300:30:133
327943994233,6cyclictest350437-21unixbench_singl20:10:272
327939994230,8cyclictest424577-21grep23:45:271
327934994233,6cyclictest414376-21apt-get23:19:590
327934994232,7cyclictest421645-21sendmail23:40:010
32794399419,7cyclictest354336-21cat20:25:012
32794399418,7cyclictest434896-21seq00:15:412
327943994132,6cyclictest365755-21unixbench_singl20:55:282
32794399411,12cyclictest440378-21cron00:35:002
327939994129,8cyclictest412270-21grep23:10:311
327946994033,4cyclictest400300-21unixbench_singl22:35:273
32794399409,7cyclictest379913-21seq21:38:252
327943994013,24cyclictest0-21swapper/200:10:242
32794399401,11cyclictest438267-21grep00:25:312
32794399401,11cyclictest337882-21memory19:35:202
327943994010,4cyclictest397337-21idleruntime-cro22:30:002
327939994034,4cyclictest398819-21kworker/u8:2+events_unbound22:32:531
327939994034,4cyclictest398819-21kworker/u8:2+events_unbound22:32:531
327934994033,4cyclictest425051-21munin-plugin-st23:50:000
327943993928,7cyclictest0-21swapper/222:30:322
327943993928,7cyclictest0-21swapper/222:30:312
327939993928,6cyclictest0-21swapper/119:25:241
327946993830,4cyclictest0-21swapper/319:50:013
32794399388,6cyclictest311ktimers/222:40:152
32794399384,7cyclictest0-21swapper/223:20:172
327943993813,22cyclictest0-21swapper/200:35:202
327943993811,3cyclictest412695-21latency_hist23:15:012
32794399381,12cyclictest389315-21wc22:05:222
32794399381,12cyclictest389315-21wc22:05:222
32794399381,11cyclictest333375-21sshd19:22:312
327943993810,3cyclictest406894-21switchtime22:55:262
327939993831,5cyclictest381561-21kworker/u8:2+events_unbound21:47:471
327934993827,7cyclictest403683-21grep22:45:260
327934993826,9cyclictest348741-21runrttasks20:05:280
327946993727,6cyclictest0-21swapper/319:15:013
32794399378,7cyclictest359523-21date20:40:002
32794399377,5cyclictest4039362chrt22:47:332
32794399372,8cyclictest347432-21cron20:05:002
32794399372,8cyclictest347432-21cron20:05:002
327943993727,6cyclictest0-21swapper/219:25:252
327943993726,7cyclictest0-21swapper/222:15:242
327943993712,4cyclictest428478-21idleruntime-cro00:00:002
327943993711,3cyclictest0-21swapper/221:10:252
32794399371,11cyclictest377336-21meminfo21:30:252
32794399371,10cyclictest388585-21cron22:05:002
327943993710,4cyclictest392027-21latency_hist22:14:592
327943993710,4cyclictest0-21swapper/223:40:172
327943993710,23cyclictest0-21swapper/223:35:012
327939993728,5cyclictest418008-21sh23:30:001
327939993727,5cyclictest0-21swapper/121:55:281
327934993727,7cyclictest407576-21sendmail23:00:010
32794399369,4cyclictest0-21swapper/200:05:192
32794399367,6cyclictest385976-21ntp_states21:55:252
327943993615,18cyclictest0-21swapper/223:50:172
327943993614,19cyclictest409721-21cut23:05:192
32794399361,11cyclictest383470-21sh21:50:002
32794399361,11cyclictest367077-21ps21:00:282
32794399361,11cyclictest367077-21ps21:00:272
32794399361,11cyclictest165-21systemd-journal22:22:512
327934993628,4cyclictest0-21swapper/020:55:250
327934993626,6cyclictest0-21swapper/019:30:020
327946993527,4cyclictest0-21swapper/320:40:013
32794399357,7cyclictest357871-21cut20:35:022
32794399357,5cyclictest0-21swapper/220:05:252
327943993526,6cyclictest311ktimers/221:05:152
327943993526,6cyclictest311ktimers/221:05:152
32794399351,9cyclictest374258-21grep21:20:252
32794399351,11cyclictest336258-21ntpq19:30:202
327939993524,7cyclictest400264-21grep22:35:271
32794399349,5cyclictest384698-21unixbench_multi21:50:252
32794399347,4cyclictest0-21swapper/220:55:012
327943993423,7cyclictest0-21swapper/221:45:012
327943993411,4cyclictest343125-21ntpq19:50:222
32794399341,12cyclictest344369-21df_inode19:55:172
32794399341,11cyclictest404707-21/usr/sbin/munin22:50:182
32794399341,10cyclictest329678-21munin-node19:15:012
327943993410,3cyclictest637-21runrttasks19:49:562
32794399340,13cyclictest429146-21meminfo00:00:222
327939993426,5cyclictest388374-21kworker/u8:0+events_unbound22:07:501
327939993426,5cyclictest388374-21kworker/u8:0+events_unbound22:07:501
327939993426,4cyclictest378352-21sh21:35:011
32793999341,5cyclictest131rcu_preempt20:40:141
327934993427,3cyclictest0-21swapper/019:45:010
327934993425,6cyclictest383483-21cut21:50:010
327934993425,4cyclictest0-21swapper/000:08:100
327934993423,6cyclictest0-21swapper/022:25:010
327946993324,4cyclictest0-21swapper/322:20:003
327946993323,6cyclictest0-21swapper/300:03:093
32794399339,7cyclictest373205-21sed21:20:002
32794399331,9cyclictest414372-21cron23:20:002
327943993313,6cyclictest311ktimers/219:40:132
327943993312,3cyclictest637-21runrttasks00:23:132
327943993312,3cyclictest637-21runrttasks00:23:122
32794399331,11cyclictest425046-21idleruntime-cro23:50:012
327943993310,5cyclictest311ktimers/220:15:262
327939993325,4cyclictest0-21swapper/123:00:011
327939993325,4cyclictest0-21swapper/100:20:001
327939993323,5cyclictest0-21swapper/121:50:241
327934993326,4cyclictest0-21swapper/019:15:000
327946993221,8cyclictest391ktimers/320:15:143
327946993221,7cyclictest0-21swapper/300:10:183
32794399323,11cyclictest375424-21memory21:25:222
32794399321,11cyclictest417168-21grep23:25:232
327939993223,5cyclictest0-21swapper/119:15:021
327939993222,5cyclictest0-21swapper/123:15:261
327939993222,5cyclictest0-21swapper/119:45:011
327934993225,3cyclictest0-21swapper/021:30:020
327934993224,3cyclictest0-21swapper/022:30:320
327934993224,3cyclictest0-21swapper/022:30:320
327946993123,4cyclictest0-21swapper/321:40:243
327946993123,4cyclictest0-21swapper/320:30:323
327946993123,4cyclictest0-21swapper/300:40:013
327946993123,4cyclictest0-21swapper/300:15:263
327946993111,4cyclictest3880402timerwakeupswit22:00:253
327943993122,6cyclictest0-21swapper/223:35:262
327939993124,4cyclictest410519-21unixbench_multi23:05:271
327939993124,4cyclictest0-21swapper/121:10:251
327939993123,4cyclictest0-21swapper/122:40:271
327939993123,4cyclictest0-21swapper/121:40:161
327939993121,6cyclictest0-21swapper/119:50:191
32793999310,5cyclictest131rcu_preempt22:10:141
327934993125,3cyclictest355970-21kthreadd20:30:000
327934993123,4cyclictest0-21swapper/021:35:240
327946993021,5cyclictest370033-21fschecks_time21:10:153
327946993021,5cyclictest0-21swapper/323:10:313
327946993020,6cyclictest0-21swapper/323:05:193
327946993019,7cyclictest398604-21grep22:30:323
327946993019,7cyclictest398604-21grep22:30:313
32793999302,4cyclictest131rcu_preempt00:35:171
327939993023,3cyclictest342324-21kthreadd19:50:011
327934993022,4cyclictest415877-21unixbench_singl23:20:260
327934993022,4cyclictest0-21swapper/023:30:180
327934993018,8cyclictest215-21dbus-daemon00:03:090
327946992923,3cyclictest0-21swapper/319:55:183
327946992921,4cyclictest0-21swapper/320:50:013
327946992921,4cyclictest0-21swapper/320:20:283
327946992921,4cyclictest0-21swapper/319:30:163
327946992920,5cyclictest391579-21unixbench_singl22:10:283
327946992920,4cyclictest0-21swapper/322:22:513
327943992922,5cyclictest355550-21unixbench_singl20:25:282
327939992921,4cyclictest0-21swapper/123:40:261
327939992921,4cyclictest0-21swapper/119:20:251
327939992920,6cyclictest393633-21kworker/u8:0+events_unbound22:19:591
327939992919,4cyclictest0-21swapper/121:25:001
327939992918,7cyclictest352096-21grep20:15:251
327939992916,8cyclictest0-21swapper/123:20:171
327934992925,2cyclictest13650irq/42-30be0000.ethernet22:40:180
327934992920,4cyclictest338471-21sshd19:35:250
327946992820,5cyclictest426282-21unixbench_multi23:50:253
327946992820,4cyclictest0-21swapper/323:25:263
327946992819,7cyclictest331828-21kworker/u8:1+events_unbound19:22:343
327946992818,7cyclictest338938-21idleruntime-cro19:40:003
327939992819,5cyclictest380058-21munin-run21:40:011
327939992818,7cyclictest428459-21run-parts00:00:001
327934992822,3cyclictest0-21swapper/020:00:270
327934992822,3cyclictest0-21swapper/020:00:270
327934992821,3cyclictest0-21swapper/021:40:180
327934992819,6cyclictest362323-21grep20:45:290
327934992817,7cyclictest0-21swapper/023:25:260
327946992717,6cyclictest397327-21sh22:29:593
327939992720,4cyclictest396443-21ntpq22:25:241
327939992720,4cyclictest0-21swapper/121:05:291
327939992720,4cyclictest0-21swapper/121:05:281
327939992718,5cyclictest0-21swapper/120:35:271
327939992716,7cyclictest364056-21grep20:50:291
32793499279,4cyclictest374776-21latency_hist21:25:000
32793499276,3cyclictest3672852expr21:00:300
32793499276,3cyclictest3672852expr21:00:300
327934992721,3cyclictest0-21swapper/000:19:590
327934992720,4cyclictest0-21swapper/019:20:250
327934992719,5cyclictest369515-21kworker/u8:0+events_unbound21:52:470
327934992719,4cyclictest412212-21tail23:10:300
327934992719,4cyclictest0-21swapper/022:25:170
327934992716,8cyclictest215-21dbus-daemon00:13:100
327946992618,4cyclictest0-21swapper/321:05:203
327946992618,4cyclictest0-21swapper/321:05:203
327946992618,3cyclictest0-21swapper/320:25:163
327939992619,3cyclictest0-21swapper/120:25:281
327939992618,5cyclictest0-21swapper/123:30:261
327939992615,7cyclictest0-21swapper/121:25:161
327934992620,3cyclictest0-21swapper/019:50:230
327934992619,4cyclictest0-21swapper/020:40:290
327934992618,4cyclictest0-21swapper/023:50:160
327934992617,7cyclictest392414-21fschecks_time22:15:150
327934992614,9cyclictest344256-21apt-get19:55:150
327946992519,3cyclictest0-21swapper/323:30:183
327946992519,3cyclictest0-21swapper/321:00:283
327946992519,3cyclictest0-21swapper/321:00:283
327946992519,3cyclictest0-21swapper/319:30:013
327946992518,4cyclictest0-21swapper/321:30:333
327946992518,3cyclictest0-21swapper/323:35:263
327946992518,3cyclictest0-21swapper/323:00:173
327946992518,3cyclictest0-21swapper/321:20:003
327946992517,4cyclictest0-21swapper/323:55:193
32793999252,5cyclictest131rcu_preempt22:50:141
327939992519,3cyclictest0-21swapper/120:30:231
327939992518,4cyclictest0-21swapper/120:45:261
327939992517,4cyclictest0-21swapper/122:20:261
327939992516,5cyclictest331871-21auditctl19:20:011
327939992515,7cyclictest0-21swapper/120:10:201
327939992515,6cyclictest231ktimers/100:10:001
327939992514,8cyclictest215-21dbus-daemon20:22:371
327934992519,3cyclictest437959-21mailstats00:25:280
327946992418,3cyclictest0-21swapper/321:50:203
327946992418,3cyclictest0-21swapper/320:55:003
327946992415,5cyclictest0-21swapper/320:05:273
327946992414,7cyclictest330961-21ntp_states19:15:213
327946992413,8cyclictest366200-21run-parts21:00:003
327946992413,7cyclictest0-21swapper/323:40:263
327946992412,9cyclictest327929-21cyclictest00:25:163
327943992418,3cyclictest0-21swapper/219:20:002
327939992417,4cyclictest429734-21grep00:00:281
327939992417,4cyclictest366191-21latency_hist21:00:011
327939992416,4cyclictest0-21swapper/120:10:001
32793999241,4cyclictest131rcu_preempt00:10:141
327934992417,4cyclictest0-21swapper/000:40:000
327934992416,5cyclictest369515-21kworker/u8:0+events_unbound21:17:420
327934992416,5cyclictest0-21swapper/021:10:160
327934992415,6cyclictest0-21swapper/000:00:010
32793499241,21cyclictest0-21swapper/022:00:180
32793499241,19cyclictest0-21swapper/022:09:180
32793499241,19cyclictest0-21swapper/022:09:170
32793499241,19cyclictest0-21swapper/020:23:370
32793499241,19cyclictest0-21swapper/019:32:200
327946992317,3cyclictest0-21swapper/323:20:003
32794699231,4cyclictest0-21swapper/322:41:363
32794399234,3cyclictest360332-21tr20:40:272
32793999236,3cyclictest131rcu_preempt23:00:131
327939992316,3cyclictest0-21swapper/121:20:001
327939992315,4cyclictest338900-21sh19:40:011
327939992311,8cyclictest440388-21audit_send_reply00:34:591
327934992316,5cyclictest0-21swapper/000:20:230
327934992316,5cyclictest0-21swapper/000:20:230
327934992316,3cyclictest0-21swapper/021:31:100
327934992316,3cyclictest0-21swapper/020:32:550
327934992316,3cyclictest0-21swapper/019:15:020
327934992313,7cyclictest341590-21tr19:45:260
327934992313,6cyclictest0-21swapper/020:18:340
32793499231,18cyclictest0-21swapper/023:00:070
32793499231,18cyclictest0-21swapper/020:52:350
32793499230,19cyclictest0-21swapper/021:55:470
327946992216,3cyclictest0-21swapper/320:10:193
327946992215,3cyclictest0-21swapper/321:45:253
327946992213,6cyclictest415879-21grep23:20:253
327946992213,6cyclictest0-21swapper/321:20:173
327939992215,3cyclictest0-21swapper/119:55:181
327939992214,5cyclictest231ktimers/119:31:521
327939992214,4cyclictest426276-21grep23:50:251
327934992216,3cyclictest0-21swapper/023:40:190
327934992215,4cyclictest439069-21fschecks_count00:30:180
327934992215,3cyclictest0-21swapper/020:10:570
32793499221,19cyclictest0-21swapper/021:09:250
32793499221,19cyclictest0-21swapper/021:09:240
32793499221,18cyclictest121ktimers/022:14:370
327946992116,3cyclictest0-21swapper/300:05:253
327946992115,3cyclictest0-21swapper/321:35:153
327946992115,3cyclictest0-21swapper/319:55:013
327946992113,3cyclictest0-21swapper/322:50:013
32793999217,9cyclictest231ktimers/122:00:121
327939992114,5cyclictest384905-21kworker/u8:1+events_unbound22:47:391
327939992112,6cyclictest346518-21ntpq20:00:231
327939992112,6cyclictest346518-21ntpq20:00:231
327934992116,3cyclictest0-21swapper/022:37:450
32794699208,7cyclictest405394-21grep22:50:253
327946992014,3cyclictest637-21runrttasks23:45:173
327946992014,3cyclictest0-21swapper/322:55:283
327946992012,4cyclictest0-21swapper/322:05:253
327946992012,4cyclictest0-21swapper/322:05:243
327946992011,6cyclictest339561-21sort19:40:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional