You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-07 - 11:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sun Dec 07, 2025 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
299299240,23cyclictest2698-21tr21:19:593
299299240,23cyclictest0-21swapper/322:10:023
2991992422,1cyclictest0-21swapper/222:00:002
299199240,23cyclictest0-21swapper/219:38:262
2990992422,1cyclictest15131-21munin-run20:40:001
299099240,23cyclictest27831-21turbostat00:20:001
2989992423,1cyclictest8520-21apt-get22:38:280
2989992421,2cyclictest7541-21munin-run21:30:010
2992992323,0cyclictest0-21swapper/323:23:253
2992992323,0cyclictest0-21swapper/320:38:273
2992992321,1cyclictest25173-21cron22:05:003
299299230,22cyclictest28585-21apt-get23:18:253
299299230,22cyclictest25631-21cron23:10:003
2990992320,2cyclictest7695-21sendmail-msp23:40:001
299099231,21cyclictest598-21rs:main1
299099230,22cyclictest596-21in:imuxsock00:15:001
299099230,22cyclictest32309-21ls21:15:001
299099230,22cyclictest17525-21perf20:44:591
298999230,2cyclictest29921-21wc21:10:000
298999230,2cyclictest25674-21idleruntime-cro23:10:000
298999230,22cyclictest593-21dbus-daemon20:00:000
298999230,22cyclictest27847-21ls00:20:000
298999230,22cyclictest0-21swapper/022:03:290
299299220,21cyclictest13459-21apt-get23:53:283
299299220,1cyclictest18464-21apt-get00:00:013
2991992222,0cyclictest0-21swapper/222:43:302
2991992222,0cyclictest0-21swapper/222:35:012
2991992220,1cyclictest0-21swapper/219:55:032
2991992219,2cyclictest596-21in:imuxsock22:15:012
299199220,21cyclictest733-21apt-get00:33:282
299199220,21cyclictest583-21avahi-daemon20:09:592
299199220,21cyclictest19974-21sh21:55:002
299199220,21cyclictest19915-21apt-get20:50:012
299199220,21cyclictest0-21swapper/221:03:282
299199220,21cyclictest0-21swapper/219:53:262
2990992219,2cyclictest25055-21munin-plugin-st19:55:011
299099220,21cyclictest5149-21munin-run21:25:001
299099220,21cyclictest28585-21apt-get23:18:271
299099220,21cyclictest16182-21apt-get22:53:271
299099220,21cyclictest10086-21latency_hist23:45:001
299099220,21cyclictest0-21swapper/122:23:281
299099220,21cyclictest0-21swapper/100:33:281
299099220,21cyclictest0-21swapper/100:28:261
299099220,1cyclictest7460-21cat20:25:001
299099220,1cyclictest12373-21idleruntime-cro21:40:001
2989992221,0cyclictest30774-21apt-get20:08:260
2989992217,4cyclictest0-21swapper/022:33:270
298999220,21cyclictest32323-21apt-get21:15:000
298999220,21cyclictest0-21swapper/022:10:000
298999220,21cyclictest0-21swapper/020:53:260
298999220,21cyclictest0-21swapper/000:08:290
298999220,1cyclictest30469-21sendmail-msp23:20:000
298999220,1cyclictest20481-21basename23:00:000
298999220,1cyclictest15032-21apt-get19:35:020
2992992121,0cyclictest0-21swapper/321:43:293
2992992121,0cyclictest0-21swapper/321:23:283
2992992121,0cyclictest0-21swapper/319:35:023
2992992120,1cyclictest12869-21turbostat22:45:003
2992992120,0cyclictest15879-21apt-get22:50:023
2992992120,0cyclictest15853-21apt-get23:58:263
299299210,20cyclictest30451-21apt-get21:13:293
299299210,20cyclictest0-21swapper/320:43:303
299299210,0cyclictest0-21swapper/319:28:273
2991992120,1cyclictest10709-21apt-get19:28:272
2991992120,0cyclictest28049-21apt-get21:08:272
2991992119,1cyclictest10780-21apt-get20:33:282
299199210,21cyclictest0-21swapper/223:50:022
299199210,21cyclictest0-21swapper/221:13:302
299199210,20cyclictest0-21swapper/223:23:262
299199210,20cyclictest0-21swapper/219:13:272
299199210,0cyclictest30770-21apt-get00:28:262
2990992121,0cyclictest28379-21apt-get20:03:271
2990992121,0cyclictest10617-21apt-get23:48:301
2990992121,0cyclictest0-21swapper/119:58:271
2990992121,0cyclictest0-21swapper/119:35:021
2990992120,0cyclictest0-21swapper/123:53:281
2990992118,2cyclictest22-21ksoftirqd/123:00:001
299099210,20cyclictest8055-21apt-get21:33:281
299099210,20cyclictest736-21apt-get20:13:281
2989992119,1cyclictest0-21swapper/021:50:000
2989992119,1cyclictest0-21swapper/021:00:000
298999210,21cyclictest0-21swapper/020:18:280
298999210,20cyclictest8313-21apt-get19:23:260
298999210,20cyclictest0-21swapper/023:53:280
298999210,20cyclictest0-21swapper/022:48:270
298999210,20cyclictest0-21swapper/021:33:270
298999210,20cyclictest0-21swapper/021:15:020
298999210,20cyclictest0-21swapper/020:23:270
298999210,0cyclictest3326-21apt-get22:28:260
298999210,0cyclictest0-21swapper/022:18:270
298999210,0cyclictest0-21swapper/020:33:270
298999210,0cyclictest0-21swapper/019:13:270
298999210,0cyclictest0-21swapper/000:33:280
2992992020,0cyclictest0-21swapper/322:48:283
2992992020,0cyclictest0-21swapper/321:48:273
2992992020,0cyclictest0-21swapper/321:03:293
2992992020,0cyclictest0-21swapper/320:18:273
2992992020,0cyclictest0-21swapper/319:53:263
2992992020,0cyclictest0-21swapper/300:38:293
2992992020,0cyclictest0-21swapper/300:18:273
2992992020,0cyclictest0-21swapper/300:13:273
2992992019,1cyclictest19904-21awk20:50:003
2992992018,1cyclictest583-21avahi-daemon21:40:003
299299200,0cyclictest25961-21apt-get19:58:273
299299200,0cyclictest0-21swapper/320:28:273
2991992020,0cyclictest5660-21apt-get21:28:282
2991992020,0cyclictest23234-21apt-get20:58:252
2991992020,0cyclictest13230-21apt-get20:38:272
2991992020,0cyclictest0-21swapper/219:33:282
2991992019,1cyclictest8313-21apt-get19:23:272
2991992019,1cyclictest12929-21apt-get21:43:292
299199200,20cyclictest0-21swapper/222:33:292
299199200,20cyclictest0-21swapper/220:25:022
299199200,20cyclictest0-21swapper/219:43:262
299199200,19cyclictest0-21swapper/223:38:272
299199200,0cyclictest0-21swapper/223:58:252
299199200,0cyclictest0-21swapper/221:33:282
299199200,0cyclictest0-21swapper/220:48:272
299199200,0cyclictest0-21swapper/220:43:302
299199200,0cyclictest0-21swapper/200:08:282
2990992020,0cyclictest0-21swapper/123:05:021
2990992020,0cyclictest0-21swapper/122:43:301
2990992020,0cyclictest0-21swapper/122:38:281
2990992020,0cyclictest0-21swapper/121:45:011
2990992020,0cyclictest0-21swapper/121:03:291
2990992019,0cyclictest0-21swapper/122:03:291
299099200,20cyclictest0-21swapper/121:28:281
299099200,20cyclictest0-21swapper/100:38:291
299099200,0cyclictest3170-21apt-get20:18:271
299099200,0cyclictest0-21swapper/123:33:271
299099200,0cyclictest0-21swapper/122:33:281
299099200,0cyclictest0-21swapper/122:18:271
2989992020,0cyclictest13230-21apt-get20:38:270
2989992020,0cyclictest0-21swapper/023:58:260
2989992020,0cyclictest0-21swapper/023:13:280
2989992020,0cyclictest0-21swapper/019:48:250
2989992020,0cyclictest0-21swapper/019:18:260
2989992017,2cyclictest20646-21awk00:05:000
298999200,19cyclictest0-21swapper/019:43:260
298999200,0cyclictest0-21swapper/022:43:300
298999200,0cyclictest0-21swapper/000:28:260
2992991919,0cyclictest909-21apt-get22:23:283
2992991919,0cyclictest0-21swapper/323:33:273
2992991919,0cyclictest0-21swapper/322:08:263
2992991919,0cyclictest0-21swapper/321:28:283
2992991919,0cyclictest0-21swapper/300:33:273
2991991919,0cyclictest30913-21apt-get22:18:272
2991991919,0cyclictest0-21swapper/223:48:312
2991991919,0cyclictest0-21swapper/223:08:252
2990991919,0cyclictest25722-21apt-get22:08:261
2990991919,0cyclictest0-21swapper/119:48:261
2990991919,0cyclictest0-21swapper/119:18:271
2990991919,0cyclictest0-21swapper/100:03:271
2990991918,1cyclictest22718-21perf20:55:001
2989991919,0cyclictest31001-21apt-get23:23:270
2989991919,0cyclictest0-21swapper/023:33:270
2989991919,0cyclictest0-21swapper/021:53:260
299299184,13cyclictest10393-21idleruntime-cro22:40:003
2992991818,0cyclictest0-21swapper/323:03:273
2992991818,0cyclictest0-21swapper/322:28:253
2992991818,0cyclictest0-21swapper/300:23:273
2992991817,0cyclictest20535-21apt-get20:50:013
2992991816,1cyclictest19980-21latency_hist21:55:003
2992991815,2cyclictest22383-21apt-get22:00:003
2991991818,0cyclictest17741-21apt-get00:00:012
2991991818,0cyclictest0-21swapper/223:13:282
2991991818,0cyclictest0-21swapper/222:05:002
2991991818,0cyclictest0-21swapper/221:23:282
2991991818,0cyclictest0-21swapper/220:13:282
2991991818,0cyclictest0-21swapper/200:23:282
2991991817,1cyclictest821-21apt-get21:18:292
2991991817,1cyclictest3398-21apt-get23:33:272
2991991817,0cyclictest3165-21apt-get00:38:292
2991991815,2cyclictest2785-21dump-pmu-power22:25:002
2990991818,0cyclictest0-21swapper/123:28:271
2990991818,0cyclictest0-21swapper/120:48:271
2990991818,0cyclictest0-21swapper/119:43:261
2990991818,0cyclictest0-21swapper/119:23:271
2990991817,1cyclictest0-21swapper/121:55:001
299099180,17cyclictest5186-21cron22:30:001
2989991818,0cyclictest0-21swapper/020:13:280
2989991816,1cyclictest15052-21munin-run19:35:000
2992991717,0cyclictest0-21swapper/323:28:273
2992991717,0cyclictest0-21swapper/323:13:283
2992991717,0cyclictest0-21swapper/319:18:263
2992991716,1cyclictest3497-21apt-get19:13:273
2992991715,1cyclictest32636-21grep20:10:003
299299170,1cyclictest12717-21munin-plugin-st20:35:003
2991991717,0cyclictest0-21swapper/223:03:262
2991991716,0cyclictest0-21swapper/222:58:292
2991991716,0cyclictest0-21swapper/222:08:252
2990991717,0cyclictest26187-21apt-get23:13:281
2990991717,0cyclictest13162-21apt-get19:33:281
2990991717,0cyclictest0-21swapper/122:48:281
2990991717,0cyclictest0-21swapper/121:43:291
2989991717,0cyclictest0-21swapper/023:38:270
2989991717,0cyclictest0-21swapper/020:25:020
2989991717,0cyclictest0-21swapper/019:28:270
2989991715,1cyclictest766-21snmpd20:43:300
2989991715,1cyclictest0-21swapper/022:00:000
298999170,0cyclictest0-21swapper/022:53:280
2992991616,0cyclictest0-21swapper/323:48:313
2992991616,0cyclictest0-21swapper/320:23:273
2992991616,0cyclictest0-21swapper/300:28:263
2992991615,1cyclictest8055-21apt-get21:33:283
2992991615,1cyclictest21163-21apt-get00:08:283
299299160,15cyclictest28793-21df20:03:293
2991991616,0cyclictest27840-21sendmail-msp00:20:002
2991991616,0cyclictest14788-21apt-get21:45:012
2991991616,0cyclictest13389-21apt-get22:48:282
2991991616,0cyclictest0-21swapper/222:53:282
2991991616,0cyclictest0-21swapper/220:18:272
299199160,15cyclictest28866-21cut20:03:302
2990991616,0cyclictest0-21swapper/121:08:261
2990991614,1cyclictest5364-21sh19:15:001
2990991614,1cyclictest2731-21sleep21:20:001
2990991614,1cyclictest0-21swapper/122:13:371
2989991616,0cyclictest0-21swapper/023:48:310
2989991616,0cyclictest0-21swapper/023:28:270
298999160,15cyclictest10623-21grep21:38:270
298999160,15cyclictest0-21swapper/022:13:280
2992991515,0cyclictest0-21swapper/320:13:283
299299150,14cyclictest0-21swapper/323:43:283
299299150,14cyclictest0-21swapper/322:18:353
2991991515,0cyclictest978-21apt-get23:28:272
2991991515,0cyclictest5896-21apt-get19:18:272
2991991514,1cyclictest20769-21apt-get19:48:252
2991991514,1cyclictest0-21swapper/223:43:272
2991991513,1cyclictest0-21swapper/200:13:342
2990991514,1cyclictest0-21swapper/120:30:061
2989991515,0cyclictest0-21swapper/020:48:270
2989991515,0cyclictest0-21swapper/020:03:280
298999150,1cyclictest0-21swapper/000:38:340
298999150,14cyclictest0-21swapper/023:01:100
2992991414,0cyclictest0-21swapper/322:56:363
2992991414,0cyclictest0-21swapper/320:58:273
299299140,14cyclictest0-21swapper/322:31:513
299299140,14cyclictest0-21swapper/321:05:053
299299140,14cyclictest0-21swapper/319:32:043
299299140,0cyclictest0-21swapper/323:36:113
299299140,0cyclictest0-21swapper/319:45:143
2991991414,0cyclictest0-21swapper/223:16:272
299199140,14cyclictest0-21swapper/221:36:482
2990991414,0cyclictest0-21swapper/100:23:281
299099140,14cyclictest0-21swapper/120:05:501
299099140,0cyclictest0-21swapper/123:23:091
299099140,0cyclictest0-21swapper/119:25:111
2989991414,0cyclictest0-21swapper/022:22:210
2989991414,0cyclictest0-21swapper/021:20:340
2989991414,0cyclictest0-21swapper/021:01:320
298999140,14cyclictest0-21swapper/023:41:260
298999140,14cyclictest0-21swapper/000:22:260
298999140,14cyclictest0-21swapper/000:12:570
298999140,0cyclictest0-21swapper/021:42:360
2992991313,0cyclictest0-21swapper/319:20:083
2991991313,0cyclictest0-21swapper/222:25:292
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional