You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 13:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot5s.osadl.org (updated Mon Dec 01, 2025 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2178990996443,17cyclictest2207030-21ls20:30:001
2178990995826,6cyclictest0-21swapper/121:25:021
2178995995225,22cyclictest0-21swapper/221:55:282
2178995995031,16cyclictest0-21swapper/221:20:002
2178995994827,17cyclictest0-21swapper/219:20:002
2178995994827,17cyclictest0-21swapper/219:20:002
2178986994739,5cyclictest2273108-21grep23:40:000
2178999994637,5cyclictest2217873-21cron21:00:003
2178999994536,4cyclictest2182928-21latency_hist19:20:003
2178999994536,4cyclictest2182928-21latency_hist19:20:003
2178995994435,6cyclictest2201896-21cron20:15:012
217899599441,12cyclictest2204800-21/usr/sbin/munin20:20:282
2178986994436,4cyclictest2205325-21cron20:25:010
2178999994335,6cyclictest391ktimers/323:55:133
2178999994334,6cyclictest2206629-21unixbench_singl20:25:273
2178999994333,6cyclictest2269706-21munin-plugin-st23:30:003
2178995994335,5cyclictest440-21cron21:50:002
2178995994335,5cyclictest440-21cron21:50:002
2178995994321,19cyclictest2196581-21munin-plugin-st20:00:012
2178995994321,19cyclictest2196581-21munin-plugin-st20:00:002
217899599430,10cyclictest2249249-21/usr/sbin/munin22:30:272
2178986994336,4cyclictest2276526-21ls23:50:000
217899599421,9cyclictest2228977-21/usr/sbin/munin21:30:202
217899599421,7cyclictest2211803-21/usr/sbin/munin20:40:272
2178995994215,23cyclictest0-21swapper/221:35:262
217899599421,11cyclictest2243633-21cron22:12:002
2178999994130,8cyclictest391ktimers/322:05:133
2178999994128,8cyclictest2282273-21cron00:10:003
217899599412,9cyclictest2207372-21df_inode20:30:162
217899599412,4cyclictest2258340-21ntp_states22:55:222
2178995994120,17cyclictest0-21swapper/220:25:182
217899599411,9cyclictest2288209-21/usr/sbin/munin00:25:202
217899599411,9cyclictest2272708-21/usr/sbin/munin23:35:272
217899599411,9cyclictest2218534-21/usr/sbin/munin21:00:192
217899599411,9cyclictest2202414-21/usr/sbin/munin20:15:182
217899599411,8cyclictest2233361-21sshd21:41:222
217899599411,10cyclictest2215070-21/usr/sbin/munin20:50:232
2178995994020,16cyclictest0-21swapper/223:25:012
217899599401,9cyclictest2274334-21/usr/sbin/munin23:40:262
217899599401,9cyclictest2274334-21/usr/sbin/munin23:40:252
217899599401,8cyclictest2284519-21/usr/sbin/munin00:15:132
217899599401,8cyclictest2226883-21/usr/sbin/munin21:25:142
217899599401,8cyclictest2188836-21/usr/sbin/munin19:35:222
217899599401,6cyclictest2268231-21apt-get23:25:122
217899599401,5cyclictest2187264-21/usr/sbin/munin19:30:242
2178995994013,24cyclictest2239343-21if_eth122:00:192
217899599401,10cyclictest2236158-21/usr/sbin/munin21:50:212
217899599400,11cyclictest2225173-21apt-get21:20:112
2178990994031,5cyclictest0-21swapper/122:05:011
217899599392,7cyclictest2279455-21grep23:55:252
217899599391,8cyclictest2262022-21/usr/sbin/munin23:05:252
217899599391,8cyclictest2245367-21sshd22:17:222
217899599391,7cyclictest2197160-21/usr/sbin/munin20:00:182
217899599391,6cyclictest2291586-21/usr/sbin/munin00:35:232
217899599391,4cyclictest2221539-21/usr/sbin/munin21:10:132
217899999383,29cyclictest215-21dbus-daemon23:17:593
2178999993831,4cyclictest2224926-21kthreadd21:20:003
2178999993830,4cyclictest2290832-21kthreadd00:35:003
2178999993828,7cyclictest2184166-21grep19:20:283
2178999993828,6cyclictest0-21swapper/323:20:163
2178999993827,7cyclictest2238378-21grep21:55:283
217899599386,27cyclictest2193565-21fschecks_count19:50:192
217899599382,4cyclictest2246524-21/usr/sbin/munin22:20:222
217899599382,11cyclictest2275170-21diskstats23:45:152
217899599381,5cyclictest2179732-21/usr/sbin/munin19:10:252
217899599381,4cyclictest2289888-21/usr/sbin/munin00:30:242
217899599381,4cyclictest2216738-21/usr/sbin/munin20:55:192
217899599381,4cyclictest2212963-21/usr/sbin/munin20:45:192
217899599381,4cyclictest2212963-21/usr/sbin/munin20:45:192
217899599381,4cyclictest2199898-21cron20:09:002
2178995993814,21cyclictest0-21swapper/200:10:212
217899599381,11cyclictest2270379-21memory23:30:192
217899599381,10cyclictest2264916-21df23:15:162
2178990993831,4cyclictest2196577-21munin-plugin-st20:00:001
2178990993831,4cyclictest2196577-21munin-plugin-st20:00:001
217899599372,4cyclictest2280746-21ntp_states00:00:222
2178995993720,14cyclictest0-21swapper/222:25:182
217899599371,8cyclictest2276935-21fschecks_time23:50:172
217899599371,6cyclictest2260079-21open_files23:00:232
217899599371,4cyclictest2263675-21/usr/sbin/munin23:10:222
217899599371,4cyclictest2254429-21/usr/sbin/munin22:45:142
217899599371,4cyclictest2192273-21ntp_states19:45:212
217899599371,3cyclictest2257152-21/usr/sbin/munin22:50:292
217899599371,3cyclictest2257152-21/usr/sbin/munin22:50:282
217899599371,10cyclictest2185418-21/usr/sbin/munin19:25:202
2178995993710,22cyclictest0-21swapper/220:35:282
2178990993725,7cyclictest2233272-21grep21:40:261
2178986993730,3cyclictest0-21swapper/023:20:000
2178986993727,6cyclictest0-21swapper/020:32:240
2178999993628,6cyclictest391ktimers/320:30:133
217899599364,4cyclictest2183413-21fw_packets19:20:192
217899599362,3cyclictest2190212-21fschecks_time19:40:182
217899599361,8cyclictest2220175-21irqstats21:05:202
217899599361,7cyclictest2282417-21latency_hist00:10:012
217899599361,7cyclictest2251707-21/usr/sbin/munin22:35:262
217899599361,4cyclictest2241532-21/usr/sbin/munin22:05:252
217899599361,3cyclictest2252783-21df22:40:152
2178990993629,3cyclictest0-21swapper/123:40:001
2178986993627,5cyclictest0-21swapper/000:30:000
217899599352,4cyclictest2286179-21/usr/sbin/munin00:20:152
2178999993425,5cyclictest53-21kcompactd023:30:263
2178990993427,3cyclictest0-21swapper/122:50:271
2178990993427,3cyclictest0-21swapper/122:50:271
2178990993426,4cyclictest0-21swapper/122:20:151
2178990993423,7cyclictest0-21swapper/123:55:251
2178986993425,5cyclictest2252440-21sendmail22:40:000
2178986993424,6cyclictest0-21swapper/000:00:000
2178999993322,7cyclictest2240095-21grep22:00:263
2178999993322,7cyclictest0-21swapper/319:50:003
2178999993322,6cyclictest0-21swapper/320:10:003
2178990993326,4cyclictest2290823-21cron00:35:001
2178990993322,8cyclictest231ktimers/120:15:131
2178986993324,6cyclictest2210442-21sendmail20:40:000
2178999993225,3cyclictest0-21swapper/300:05:013
2178999993224,5cyclictest2246756-21kworker/u8:0+events_unbound22:22:493
2178999993221,7cyclictest0-21swapper/321:45:263
2178999993221,7cyclictest0-21swapper/321:45:263
2178999993221,5cyclictest0-21swapper/321:40:163
2178990993226,3cyclictest0-21swapper/120:55:011
2178990993224,4cyclictest0-21swapper/100:35:281
2178990993223,6cyclictest2270973-21unixbench_singl23:30:261
2178986993225,4cyclictest0-21swapper/020:00:000
2178986993225,4cyclictest0-21swapper/020:00:000
2178986993223,5cyclictest2268393-21fschecks_count23:25:160
2178999993123,4cyclictest0-21swapper/323:05:283
2178999993123,4cyclictest0-21swapper/319:50:283
2178990993123,5cyclictest0-21swapper/122:00:001
2178986993124,4cyclictest2240488-21cron22:05:010
2178986993124,3cyclictest0-21swapper/023:15:000
2178986993123,4cyclictest0-21swapper/019:35:000
2178999993023,4cyclictest0-21swapper/320:05:003
2178999993022,4cyclictest0-21swapper/322:40:273
2178999993020,6cyclictest0-21swapper/300:20:143
2178999993019,7cyclictest0-21swapper/300:15:233
2178999993015,10cyclictest215-21dbus-daemon00:13:213
217899599303,6cyclictest2179012-21latency_hist19:10:012
2178990993023,4cyclictest2219594-21cron21:05:001
2178990993023,4cyclictest0-21swapper/123:55:001
2178990993021,6cyclictest2205316-21cron20:25:011
2178990993020,6cyclictest2192701-21grep19:45:241
2178986993024,3cyclictest0-21swapper/023:45:000
2178986993024,3cyclictest0-21swapper/023:45:000
2178986993024,3cyclictest0-21swapper/022:20:010
2178986993023,3cyclictest0-21swapper/023:25:000
2178986993023,3cyclictest0-21swapper/021:50:270
2178999992922,4cyclictest2214078-21idleruntime-cro20:50:003
2178999992922,4cyclictest2214078-21idleruntime-cro20:50:003
2178999992921,4cyclictest0-21swapper/320:40:283
2178999992921,4cyclictest0-21swapper/319:10:273
2178999992918,7cyclictest0-21swapper/320:35:183
2178990992922,4cyclictest0-21swapper/121:35:171
2178990992920,6cyclictest0-21swapper/123:30:011
2178990992919,6cyclictest0-21swapper/121:45:261
2178990992919,6cyclictest0-21swapper/121:45:261
2178986992919,7cyclictest121ktimers/022:40:140
2178986992919,6cyclictest0-21swapper/021:20:000
2178986992916,8cyclictest215-21dbus-daemon22:32:490
2178999992820,5cyclictest2264110-21unixbench_multi23:10:263
2178999992820,4cyclictest0-21swapper/323:00:273
2178990992820,5cyclictest0-21swapper/121:50:261
2178990992820,4cyclictest0-21swapper/121:10:161
2178999992721,4cyclictest2248657-21kworker/u8:1+events_unbound22:52:553
2178999992721,4cyclictest2248657-21kworker/u8:1+events_unbound22:52:543
2178999992719,4cyclictest0-21swapper/321:05:273
2178999992718,6cyclictest0-21swapper/323:45:163
2178990992719,5cyclictest2214057-21apt20:50:001
2178990992719,5cyclictest2214057-21apt20:50:001
2178990992719,4cyclictest0-21swapper/123:20:251
2178990992719,4cyclictest0-21swapper/121:25:261
2178990992718,5cyclictest0-21swapper/123:05:271
2178990992717,6cyclictest0-21swapper/120:05:151
2178986992717,7cyclictest2219977-21fschecks_count21:05:180
2178986992717,7cyclictest2214200-21sed20:50:010
2178986992717,7cyclictest2214200-21sed20:50:010
2178999992619,5cyclictest2252205-21kworker/u8:3+events_unbound22:47:543
2178999992618,6cyclictest2250403-21kworker/u8:2+events_unbound22:32:503
2178999992618,4cyclictest0-21swapper/300:35:193
2178999992617,6cyclictest0-21swapper/323:35:283
2178999992617,5cyclictest0-21swapper/320:20:013
2178990992618,4cyclictest2182499-21head19:15:271
2178990992618,4cyclictest2182499-21head19:15:271
2178990992618,4cyclictest0-21swapper/119:20:281
2178990992617,6cyclictest0-21swapper/100:30:001
2178990992616,6cyclictest0-21swapper/122:30:001
2178990992616,6cyclictest0-21swapper/122:15:271
2178990992615,7cyclictest2217873-21cron21:00:001
2178986992618,5cyclictest0-21swapper/000:05:000
2178986992615,8cyclictest2237429-21seq21:55:180
2178986992615,7cyclictest2285699-21cron00:20:000
2178999992520,3cyclictest2229624-21unixbench_singl21:30:283
2178999992518,4cyclictest2185924-21head19:25:253
2178999992518,4cyclictest0-21swapper/321:40:013
2178999992518,4cyclictest0-21swapper/319:45:013
2178999992517,4cyclictest0-21swapper/322:55:223
2178999992517,4cyclictest0-21swapper/319:35:173
2178990992519,3cyclictest0-21swapper/123:05:001
2178990992519,3cyclictest0-21swapper/100:10:131
2178986992520,2cyclictest13650irq/42-30be0000.ethernet20:40:050
2178986992518,4cyclictest2223202-21latency_hist21:15:010
2178986992518,4cyclictest0-21swapper/021:40:260
2178986992518,4cyclictest0-21swapper/019:40:000
2178986992518,3cyclictest0-21swapper/019:14:470
2178986992517,5cyclictest0-21swapper/022:55:260
2178986992516,5cyclictest0-21swapper/021:00:000
2178986992514,8cyclictest2206121-21cat20:25:220
2178986992510,4cyclictest121ktimers/022:30:010
2178999992417,4cyclictest0-21swapper/319:55:253
2178999992417,4cyclictest0-21swapper/319:55:253
2178999992417,3cyclictest0-21swapper/323:45:003
2178999992417,3cyclictest0-21swapper/323:45:003
2178999992414,6cyclictest0-21swapper/321:20:193
2178999992414,6cyclictest0-21swapper/300:25:163
2178990992418,3cyclictest0-21swapper/123:40:161
2178990992418,3cyclictest0-21swapper/123:40:161
2178990992418,3cyclictest0-21swapper/121:30:241
2178990992418,3cyclictest0-21swapper/120:40:011
2178990992418,3cyclictest0-21swapper/119:30:261
2178990992418,3cyclictest0-21swapper/100:20:211
2178990992414,6cyclictest0-21swapper/120:10:231
2178986992418,3cyclictest0-21swapper/020:55:000
2178986992418,3cyclictest0-21swapper/020:05:200
2178986992418,3cyclictest0-21swapper/000:20:180
2178986992417,3cyclictest0-21swapper/022:47:540
2178986992416,6cyclictest121ktimers/023:05:150
2178986992416,4cyclictest0-21swapper/023:50:260
2178986992413,8cyclictest2289451-21awk00:30:160
2178986992412,8cyclictest0-21swapper/019:25:140
217898699241,19cyclictest0-21swapper/022:21:470
217898699241,19cyclictest0-21swapper/021:34:400
217898699241,19cyclictest0-21swapper/020:15:000
217898699241,19cyclictest0-21swapper/020:00:100
2178999992314,4cyclictest0-21swapper/321:25:173
2178999992312,7cyclictest0-21swapper/322:15:273
2178990992317,3cyclictest0-21swapper/122:55:231
2178990992316,5cyclictest0-21swapper/100:15:191
2178990992316,4cyclictest2180606-21cron19:15:001
2178990992315,5cyclictest0-21swapper/122:40:271
2178990992315,4cyclictest0-21swapper/122:10:261
2178990992315,3cyclictest0-21swapper/119:55:011
2178990992313,7cyclictest2251101-21cut22:35:181
2178986992317,3cyclictest0-21swapper/019:50:150
2178986992316,5cyclictest0-21swapper/022:05:190
2178986992316,3cyclictest0-21swapper/023:00:200
2178986992316,3cyclictest0-21swapper/021:26:330
2178986992315,6cyclictest121ktimers/000:10:080
2178986992314,5cyclictest0-21swapper/000:35:420
217898699231,20cyclictest0-21swapper/019:20:160
217898699231,18cyclictest0-21swapper/021:21:270
217898699231,18cyclictest0-21swapper/019:46:010
217898699231,18cyclictest0-21swapper/000:07:390
217898699230,19cyclictest0-21swapper/022:51:060
217898699230,19cyclictest0-21swapper/022:51:060
217899999222,13cyclictest2215518-21tail20:50:273
217899999221,5cyclictest0-21swapper/322:27:563
2178999992215,5cyclictest391ktimers/320:20:153
2178990992214,5cyclictest231ktimers/122:05:211
2178990992214,5cyclictest231ktimers/120:40:151
2178990992214,5cyclictest0-21swapper/120:30:271
2178990992213,6cyclictest0-21swapper/123:20:001
217898699224,9cyclictest0-21swapper/023:31:530
2178986992216,3cyclictest0-21swapper/022:10:180
2178986992215,3cyclictest0-21swapper/021:35:420
217898699221,19cyclictest0-21swapper/019:15:180
217898699221,19cyclictest0-21swapper/019:15:180
217898699220,18cyclictest0-21swapper/021:46:310
217898699220,18cyclictest0-21swapper/021:46:310
217898699220,18cyclictest0-21swapper/021:03:320
217898699220,18cyclictest0-21swapper/019:44:320
217899999212,13cyclictest2201472-21head20:10:293
217899999211,15cyclictest2221743-21tune2fs21:10:173
2178990992115,3cyclictest0-21swapper/123:45:201
2178990992114,4cyclictest0-21swapper/119:30:001
2178990992113,5cyclictest231ktimers/121:05:201
2178990992111,7cyclictest2254503-21forks22:45:161
217899099211,17cyclictest2223459-21taskset21:15:141
2178990992110,7cyclictest2191053-21grep19:40:281
2178986992114,3cyclictest0-21swapper/020:18:150
217899999202,12cyclictest2276892-21fschecks_count23:50:173
217899999202,12cyclictest2251960-21grep22:35:273
2178999992013,5cyclictest391ktimers/322:10:133
2178999992013,5cyclictest391ktimers/321:03:003
2178999992013,5cyclictest2173444-21kworker/u8:0+events_unbound19:31:023
2178990992015,3cyclictest2281654-21munin-plugin-st00:05:001
2178990992014,4cyclictest231ktimers/123:15:001
2178990992013,3cyclictest0-21swapper/100:10:001
217899099198,7cyclictest2249823-21ntpq22:30:221
2178990991812,4cyclictest231ktimers/120:00:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional