You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-15 - 04:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Mon Sep 15, 2025 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2774399240,22cyclictest15629-21munin-run00:20:010
27754992323,0cyclictest830-21apt-get19:21:143
27754992321,1cyclictest14190-21apt23:10:013
2775499230,22cyclictest31388-21apt-get22:40:003
2775499230,22cyclictest0-21swapper/323:01:143
2775199230,22cyclictest4581-21latency_hist20:35:002
27747992323,0cyclictest23555-21apt-get00:36:171
27747992321,1cyclictest28592-21apt-get22:35:021
27747992321,1cyclictest0-21swapper/123:15:001
27747992320,2cyclictest2093-21cron21:40:011
27747992320,2cyclictest2093-21cron21:40:001
2774799230,22cyclictest16870-21apt-get22:11:131
27754992221,1cyclictest0-21swapper/320:25:023
27754992220,1cyclictest32022-21apt-get22:40:023
27754992220,1cyclictest27645-21apt-get19:10:023
2775499220,21cyclictest2468-21apt-get21:41:143
2775499220,21cyclictest15969-21apt-get00:21:143
2775499220,21cyclictest12853-21apt-get19:46:133
2775499220,21cyclictest0-21swapper/321:36:153
2775499220,21cyclictest0-21swapper/321:36:153
2775499220,21cyclictest0-21swapper/300:01:143
27751992220,1cyclictest9359-21cron20:45:002
2775199220,21cyclictest612-21gdbus21:40:002
2775199220,21cyclictest612-21gdbus21:40:002
2775199220,21cyclictest22175-21ls20:05:012
2775199220,21cyclictest14097-21cron22:05:002
2775199220,1cyclictest9368-21ls23:00:002
2775199220,1cyclictest0-21swapper/200:01:142
27747992221,1cyclictest0-21swapper/121:56:151
27747992220,1cyclictest6360-21apt-get00:01:131
27747992220,1cyclictest0-21swapper/100:10:001
2774799221,20cyclictest0-21swapper/123:50:001
2774799220,21cyclictest26955-21apt20:15:011
2774799220,21cyclictest25259-21apt-get21:21:141
2774799220,21cyclictest0-21swapper/121:41:151
2774799220,21cyclictest0-21swapper/119:21:131
2774799220,1cyclictest7689-21perf19:34:591
2774799220,0cyclictest0-21swapper/123:51:151
27743992220,1cyclictest593-21dbus-daemon00:30:000
27743992220,1cyclictest0-21swapper/000:15:010
2774399220,21cyclictest766-21snmpd23:41:160
2774399220,21cyclictest19701-21cron20:00:010
2774399220,21cyclictest0-21swapper/023:36:150
2774399220,21cyclictest0-21swapper/023:21:150
2774399220,21cyclictest0-21swapper/020:11:130
2774399220,1cyclictest9739-21apt-get23:01:130
2774399220,1cyclictest313-21systemd-journal00:00:000
2774399220,1cyclictest2520-21apt-get20:31:130
2774399220,1cyclictest0-21swapper/019:40:000
27754992120,1cyclictest0-21swapper/320:01:153
27754992120,0cyclictest0-21swapper/320:16:133
2775499210,21cyclictest0-21swapper/322:51:143
2775499210,21cyclictest0-21swapper/321:26:153
2775499210,20cyclictest9734-21apt-get20:46:133
2775499210,20cyclictest5913-21cron00:00:013
2775499210,20cyclictest20381-21apt-get21:11:083
2775499210,20cyclictest0-21swapper/323:36:143
2775499210,20cyclictest0-21swapper/322:46:143
2775499210,1cyclictest20763-21apt-get00:31:153
2775499210,1cyclictest19380-21cron21:05:003
2775499210,0cyclictest1444-21apt-get23:51:143
2775499210,0cyclictest0-21swapper/322:26:153
2775499210,0cyclictest0-21swapper/300:36:163
27751992121,0cyclictest0-21swapper/221:21:152
27751992121,0cyclictest0-21swapper/220:26:132
27751992120,1cyclictest14527-21apt-get20:56:142
27751992120,1cyclictest10801-21apt00:10:002
27751992119,1cyclictest27309-21apt-get20:16:152
2775199210,21cyclictest0-21swapper/222:51:152
2775199210,21cyclictest0-21swapper/222:11:142
2775199210,20cyclictest0-21swapper/221:35:002
2775199210,1cyclictest593-21dbus-daemon00:00:012
27747992121,0cyclictest0-21swapper/121:26:141
27747992121,0cyclictest0-21swapper/100:26:161
27747992119,1cyclictest4545-21cron22:50:001
27747992119,1cyclictest16927-21apt-get23:16:151
2774799210,21cyclictest0-21swapper/123:41:161
2774799210,20cyclictest0-21swapper/121:31:141
2774799210,20cyclictest0-21swapper/120:16:151
2774799210,20cyclictest0-21swapper/100:21:141
2774799210,1cyclictest0-21swapper/123:06:141
2774799210,0cyclictest0-21swapper/122:31:131
27743992120,1cyclictest26197-21apt22:30:010
27743992120,1cyclictest0-21swapper/019:21:130
27743992119,2cyclictest11719-21awk21:59:590
2774399210,21cyclictest0-21swapper/023:11:140
2774399210,20cyclictest0-21swapper/022:01:130
2774399210,0cyclictest0-21swapper/022:46:140
2774399210,0cyclictest0-21swapper/020:46:140
27754992020,0cyclictest0-21swapper/323:31:143
27754992020,0cyclictest0-21swapper/319:51:133
27754992020,0cyclictest0-21swapper/319:26:133
27754992020,0cyclictest0-21swapper/300:11:163
2775499201,9cyclictest0-21swapper/323:11:143
27754992019,0cyclictest7336-21apt-get20:41:143
2775499200,2cyclictest29072-21apt23:41:163
2775499200,20cyclictest0-21swapper/323:46:143
2775499200,0cyclictest0-21swapper/322:01:143
2775499200,0cyclictest0-21swapper/320:06:123
2775499200,0cyclictest0-21swapper/300:16:153
2775499200,0cyclictest0-21swapper/300:06:143
27751992020,0cyclictest0-21swapper/223:31:132
27751992020,0cyclictest0-21swapper/223:16:152
27751992020,0cyclictest0-21swapper/223:06:152
27751992020,0cyclictest0-21swapper/222:16:162
27751992020,0cyclictest0-21swapper/220:46:132
27751992020,0cyclictest0-21swapper/219:10:022
27751992019,1cyclictest16576-21turbostat23:15:012
27751992018,1cyclictest9323-21munin-plugin-st21:55:012
2775199200,20cyclictest15250-21apt-get19:51:132
2775199200,0cyclictest0-21swapper/223:01:142
2775199200,0cyclictest0-21swapper/200:36:172
27747992020,0cyclictest0-21swapper/122:51:151
27747992020,0cyclictest0-21swapper/121:16:131
27747992020,0cyclictest0-21swapper/120:06:141
27747992020,0cyclictest0-21swapper/119:26:121
27747992020,0cyclictest0-21swapper/119:11:131
27747992018,1cyclictest16524-21latency_hist22:09:591
27747992018,1cyclictest13209-21perf00:15:001
2774799200,20cyclictest0-21swapper/120:31:131
2774799200,0cyclictest19267-21apt-get22:16:141
2774799200,0cyclictest0-21swapper/120:26:121
27743992020,0cyclictest4920-21apt-get22:51:140
27743992020,0cyclictest4861-21apt-get21:46:130
27743992020,0cyclictest3240-21apt-get19:26:120
27743992020,0cyclictest0-21swapper/023:26:150
27743992020,0cyclictest0-21swapper/021:51:130
27743992020,0cyclictest0-21swapper/021:31:150
27743992020,0cyclictest0-21swapper/020:55:010
27743992020,0cyclictest0-21swapper/020:21:110
27743992020,0cyclictest0-21swapper/000:06:130
27743992019,0cyclictest4919-21apt-get20:36:170
2774399200,0cyclictest0-21swapper/020:01:160
27754991917,1cyclictest7698-21grep19:35:003
27754991917,1cyclictest0-21swapper/322:06:133
27751991919,0cyclictest8042-21apt-get19:36:142
27751991919,0cyclictest29212-21apt-get22:35:012
27751991919,0cyclictest22837-21apt-get21:16:142
27751991919,0cyclictest0-21swapper/223:26:132
27751991919,0cyclictest0-21swapper/222:21:142
27751991919,0cyclictest0-21swapper/222:06:132
27751991919,0cyclictest0-21swapper/219:26:132
27751991919,0cyclictest0-21swapper/219:21:142
27751991918,0cyclictest11155-21apt-get00:11:152
27751991917,1cyclictest19725-21munin-run20:00:002
27747991919,0cyclictest0-21swapper/120:56:141
27747991919,0cyclictest0-21swapper/120:36:161
27747991918,1cyclictest0-21swapper/121:00:021
27747991918,1cyclictest0-21swapper/120:51:141
27747991917,2cyclictest20071-21apt-get20:01:161
27747991917,1cyclictest12506-21grep19:44:591
27743991919,0cyclictest0-21swapper/023:06:140
27743991919,0cyclictest0-21swapper/021:41:150
27743991919,0cyclictest0-21swapper/019:46:140
27743991919,0cyclictest0-21swapper/019:41:150
27743991918,1cyclictest32526-21apt-get20:26:120
27743991918,1cyclictest0-21swapper/021:21:140
27754991818,0cyclictest0-21swapper/321:31:153
27754991818,0cyclictest0-21swapper/320:36:163
27754991818,0cyclictest0-21swapper/319:41:143
27751991818,0cyclictest17339-21apt-get21:01:132
27751991818,0cyclictest0-21swapper/223:41:162
27751991818,0cyclictest0-21swapper/223:21:142
27751991818,0cyclictest0-21swapper/221:56:152
27751991818,0cyclictest0-21swapper/219:41:142
27751991818,0cyclictest0-21swapper/219:16:162
27751991816,1cyclictest15630-21awk00:20:002
27747991818,0cyclictest0-21swapper/122:26:141
27747991818,0cyclictest0-21swapper/120:41:131
27747991818,0cyclictest0-21swapper/119:56:151
27747991818,0cyclictest0-21swapper/119:46:141
2774799180,13cyclictest21798-21apt-get23:26:141
27743991818,0cyclictest5641-21apt-get19:31:130
27743991818,0cyclictest0-21swapper/022:35:020
27743991818,0cyclictest0-21swapper/022:21:150
27743991818,0cyclictest0-21swapper/021:11:080
27743991818,0cyclictest0-21swapper/000:05:000
27743991817,1cyclictest20020-21ls21:09:590
27743991817,0cyclictest0-21swapper/022:16:140
27743991817,0cyclictest0-21swapper/019:16:150
2774399180,17cyclictest0-21swapper/022:41:120
2774399180,17cyclictest0-21swapper/022:14:590
27754991717,0cyclictest0-21swapper/321:46:123
27754991717,0cyclictest0-21swapper/320:21:123
27754991717,0cyclictest0-21swapper/319:56:133
27754991716,1cyclictest16870-21apt-get22:11:143
27751991717,0cyclictest0-21swapper/223:51:152
2775199170,16cyclictest0-21swapper/200:31:162
27747991717,0cyclictest26644-21apt-get23:36:161
27747991717,0cyclictest13550-21apt-get00:16:151
27747991717,0cyclictest0-21swapper/122:40:031
27747991717,0cyclictest0-21swapper/122:01:131
27747991716,1cyclictest0-21swapper/121:46:131
2774799170,14cyclictest583-21avahi-daemon23:25:001
27743991717,0cyclictest0-21swapper/023:51:140
27743991717,0cyclictest0-21swapper/023:34:010
27743991717,0cyclictest0-21swapper/023:16:160
27743991715,2cyclictest22793-21sh00:35:000
2774399170,16cyclictest28608-21munin-run22:35:010
27754991615,1cyclictest26536-21apt-get22:31:133
2775499160,15cyclictest19084-21latency_hist23:20:013
2775499160,0cyclictest18364-21apt-get00:26:143
27751991616,0cyclictest0-21swapper/223:50:002
27751991616,0cyclictest0-21swapper/223:36:142
27751991616,0cyclictest0-21swapper/222:31:132
27751991616,0cyclictest0-21swapper/219:46:122
27751991616,0cyclictest0-21swapper/200:21:142
2775199160,15cyclictest1799-21munin-run22:45:002
2775199160,10cyclictest4686-21cat21:45:002
27747991616,0cyclictest0-21swapper/119:36:141
27747991615,0cyclictest9734-21apt-get20:46:131
27747991614,1cyclictest5441-21timerwakeupswit23:56:211
27747991614,1cyclictest0-21swapper/121:11:151
27743991616,0cyclictest0-21swapper/021:16:120
27743991616,0cyclictest0-21swapper/019:11:130
27743991614,1cyclictest0-21swapper/021:01:180
2774399160,15cyclictest28367-21ntp_states20:16:190
27754991515,0cyclictest0-21swapper/322:56:133
27754991515,0cyclictest0-21swapper/322:16:153
27754991514,0cyclictest12129-21apt-get20:51:133
2775199157,8cyclictest1686-21apache221:14:012
27751991515,0cyclictest0-21swapper/222:46:152
27751991515,0cyclictest0-21swapper/220:51:132
27751991515,0cyclictest0-21swapper/220:11:142
27747991515,0cyclictest0-21swapper/121:51:141
27747991513,1cyclictest21928-21expr22:21:151
27743991515,0cyclictest0-21swapper/022:06:120
27743991515,0cyclictest0-21swapper/020:56:140
27743991515,0cyclictest0-21swapper/000:35:010
27743991515,0cyclictest0-21swapper/000:21:150
27754991414,0cyclictest0-21swapper/323:21:533
27754991414,0cyclictest0-21swapper/321:51:233
27754991414,0cyclictest0-21swapper/321:17:013
2775499140,14cyclictest0-21swapper/323:25:103
2775499140,14cyclictest0-21swapper/322:20:023
2775499140,14cyclictest0-21swapper/321:21:033
2775499140,14cyclictest0-21swapper/320:55:273
2775499140,14cyclictest0-21swapper/320:31:323
2775499140,14cyclictest0-21swapper/320:10:243
2775499140,0cyclictest0-21swapper/321:56:283
2775499140,0cyclictest0-21swapper/321:05:163
2775499140,0cyclictest0-21swapper/319:37:473
2775499140,0cyclictest0-21swapper/319:19:523
27751991414,0cyclictest27657-21apt-get21:26:142
27751991414,0cyclictest0-21swapper/220:07:302
2775199140,14cyclictest0-21swapper/221:46:242
2775199140,14cyclictest0-21swapper/221:05:482
2775199140,14cyclictest0-21swapper/220:36:042
2775199140,14cyclictest0-21swapper/220:20:402
2775199140,14cyclictest0-21swapper/219:30:392
2775199140,14cyclictest0-21swapper/200:27:402
2775199140,0cyclictest0-21swapper/222:25:332
27747991414,0cyclictest0-21swapper/100:31:541
2774799140,5cyclictest0-21swapper/119:51:141
2774799140,14cyclictest0-21swapper/123:30:051
2774799140,14cyclictest0-21swapper/123:01:001
2774799140,14cyclictest0-21swapper/122:55:391
2774799140,14cyclictest0-21swapper/121:05:081
2774799140,14cyclictest0-21swapper/120:23:201
2774799140,0cyclictest0-21swapper/119:15:191
27743991414,0cyclictest0-21swapper/023:49:050
27743991414,0cyclictest0-21swapper/022:56:190
2774399140,14cyclictest0-21swapper/019:51:520
2774399140,0cyclictest0-21swapper/021:35:430
2774399140,0cyclictest0-21swapper/021:35:430
2774399140,0cyclictest0-21swapper/021:27:170
2774399140,0cyclictest0-21swapper/020:41:170
2774399140,0cyclictest0-21swapper/020:05:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional