You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 20:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Thu Dec 04, 2025 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1464999250,24cyclictest30266-21apt-get09:58:153
1464499240,23cyclictest9308-21apt-get08:08:122
14642992421,2cyclictest12187-21cut10:25:001
14649992322,0cyclictest15080-21apt-get10:33:123
14649992321,1cyclictest0-21swapper/312:20:003
1464499230,22cyclictest24198-21perf10:50:012
1464299231,21cyclictest7361-21perf10:15:001
1464299230,22cyclictest26654-21munin-run10:55:011
14637992323,0cyclictest0-21swapper/010:13:130
14637992321,1cyclictest0-21swapper/009:48:140
1463799230,22cyclictest24226-21idleruntime-cro07:30:000
14649992222,0cyclictest0-21swapper/312:08:143
14649992220,1cyclictest0-21swapper/308:05:003
1464999220,21cyclictest23602-21latency_hist08:35:003
1464999220,1cyclictest7931-21apt-get12:30:023
14644992220,1cyclictest8667-21munin-run11:25:002
14644992220,1cyclictest5515-21cron12:25:012
14644992220,1cyclictest15308-21apt09:25:002
1464499220,21cyclictest12674-21apt-get10:28:142
14642992222,0cyclictest0-21swapper/109:03:141
14642992220,1cyclictest24737-21apt-get07:33:121
14642992220,1cyclictest0-21swapper/111:15:001
1464299220,22cyclictest0-21swapper/111:58:141
1464299220,21cyclictest8650-21turbostat11:25:001
1464299220,21cyclictest6912-21apt-get08:03:131
1464299220,21cyclictest29032-21perf11:00:001
1464299220,21cyclictest20082-21awk09:35:001
1464299220,21cyclictest17526-21apt-get07:18:141
1464299220,21cyclictest14496-21apt-get08:18:121
1464299220,21cyclictest0-21swapper/112:10:021
1464299220,21cyclictest0-21swapper/107:43:121
14637992220,1cyclictest5553-21munin-run12:25:010
14637992220,1cyclictest19400-21munin-plugin-st10:40:000
14637992219,2cyclictest21194-21cron08:30:000
14637992219,2cyclictest14499-21runrttasks08:18:110
14637992213,9cyclictest0-21swapper/009:53:130
1463799220,21cyclictest3069-21apt-get10:08:160
1463799220,21cyclictest26592-21apt-get08:43:130
1463799220,21cyclictest13952-21apt-get11:38:140
1463799220,21cyclictest0-21swapper/011:53:170
1463799220,21cyclictest0-21swapper/010:45:000
1463799220,21cyclictest0-21swapper/009:58:150
1463799220,21cyclictest0-21swapper/008:48:130
1463799220,21cyclictest0-21swapper/007:13:130
1463799220,1cyclictest8835-21apt-get12:33:140
1463799220,1cyclictest313-21systemd-journal11:10:000
1463799220,1cyclictest27938-21ls12:05:000
1463799220,1cyclictest0-21swapper/012:08:150
14649992120,1cyclictest11705-21apt-get08:13:153
14649992120,1cyclictest0-21swapper/307:28:133
14649992120,0cyclictest0-21swapper/308:43:133
14649992119,1cyclictest0-21swapper/310:05:023
1464999210,20cyclictest0-21swapper/311:58:143
1464999210,20cyclictest0-21swapper/307:58:143
1464999210,20cyclictest0-21swapper/307:38:133
1464999210,0cyclictest0-21swapper/308:18:113
14644992121,0cyclictest11557-21apt-get11:33:152
14644992120,0cyclictest0-21swapper/207:18:142
14644992119,1cyclictest612-21gdbus08:35:002
14644992119,1cyclictest313-21systemd-journal09:50:002
1464499210,21cyclictest0-21swapper/210:05:022
1464499210,21cyclictest0-21swapper/209:38:152
1464499210,20cyclictest0-21swapper/210:03:142
1464499210,0cyclictest6041-21apt-get12:28:142
1464499210,0cyclictest27869-21apt-get09:53:132
1464499210,0cyclictest0-21swapper/211:13:132
1464499210,0cyclictest0-21swapper/209:28:162
1464499210,0cyclictest0-21swapper/208:13:152
14642992120,1cyclictest32683-21apt-get10:03:151
14642992120,1cyclictest0-21swapper/111:48:171
14642992120,0cyclictest9160-21apt-get11:28:161
14642992120,0cyclictest17481-21apt-get10:38:151
14642992119,1cyclictest4495-21apt-get07:58:141
14642992119,1cyclictest24970-21perf09:45:001
14642992119,1cyclictest13594-21latency_hist08:15:001
1464299210,20cyclictest0-21swapper/112:28:141
1464299210,20cyclictest0-21swapper/109:58:151
1464299210,0cyclictest25470-21apt-get09:48:141
1464299210,0cyclictest0-21swapper/107:13:131
14637992121,0cyclictest0-21swapper/011:33:150
14637992121,0cyclictest0-21swapper/010:03:150
14637992120,0cyclictest22294-21apt-get10:48:140
14637992120,0cyclictest0-21swapper/011:03:140
14637992120,0cyclictest0-21swapper/008:53:140
14637992120,0cyclictest0-21swapper/007:43:120
1463799210,20cyclictest0-21swapper/012:38:150
1463799210,20cyclictest0-21swapper/009:33:140
1463799210,20cyclictest0-21swapper/008:13:150
1463799210,1cyclictest0-21swapper/010:29:590
1463799210,0cyclictest0-21swapper/011:43:140
1463799210,0cyclictest0-21swapper/011:28:150
1463799210,0cyclictest0-21swapper/008:35:000
1463799210,0cyclictest0-21swapper/007:53:150
14649992020,0cyclictest0-21swapper/312:38:153
14649992020,0cyclictest0-21swapper/312:03:143
14649992020,0cyclictest0-21swapper/310:43:143
14649992020,0cyclictest0-21swapper/310:38:153
14649992020,0cyclictest0-21swapper/309:43:153
14649992020,0cyclictest0-21swapper/309:18:123
14649992020,0cyclictest0-21swapper/307:13:133
14649992019,1cyclictest24692-21apt-get10:53:153
14649992019,1cyclictest15318-21perf09:24:593
14649992018,1cyclictest29710-21apt-get07:43:123
14649992018,1cyclictest18275-21wc11:45:003
14649992018,1cyclictest1597-21munin-run07:50:003
14649992018,1cyclictest15880-21perf11:40:013
14649992017,2cyclictest859-21perf08:55:003
1464999200,20cyclictest0-21swapper/310:13:123
1464999200,20cyclictest0-21swapper/308:28:133
1464999200,20cyclictest0-21swapper/308:23:133
1464999200,0cyclictest30325-21apt-get12:10:023
1464999200,0cyclictest0-21swapper/311:28:163
1464999200,0cyclictest0-21swapper/310:18:153
1464999200,0cyclictest0-21swapper/309:08:133
14644992020,0cyclictest4343-21apt-get11:18:142
14644992020,0cyclictest31235-21apt-get12:13:152
14644992020,0cyclictest23021-21apt-get09:43:152
14644992020,0cyclictest16380-21apt-get11:43:142
14644992020,0cyclictest0-21swapper/212:18:152
14644992020,0cyclictest0-21swapper/211:28:162
14644992020,0cyclictest0-21swapper/208:15:022
14644992020,0cyclictest0-21swapper/208:03:112
14644992020,0cyclictest0-21swapper/207:13:132
1464499200,20cyclictest0-21swapper/207:28:132
1464499200,0cyclictest21181-21apt-get11:53:172
1464499200,0cyclictest19897-21apt-get10:43:132
1464499200,0cyclictest0-21swapper/212:40:002
14642992020,0cyclictest8586-21apt-get09:13:151
14642992020,0cyclictest0-21swapper/112:23:141
14642992020,0cyclictest0-21swapper/110:05:021
14642992020,0cyclictest0-21swapper/109:08:131
14642992020,0cyclictest0-21swapper/108:48:141
14642992018,1cyclictest11235-21apt-get12:38:141
1464299200,20cyclictest0-21swapper/108:38:141
1464299200,20cyclictest0-21swapper/108:08:121
1464299200,0cyclictest0-21swapper/110:43:131
1464299200,0cyclictest0-21swapper/110:33:121
1464299200,0cyclictest0-21swapper/109:53:121
14637992020,0cyclictest0-21swapper/012:28:130
14637992020,0cyclictest0-21swapper/011:48:170
14637992020,0cyclictest0-21swapper/008:03:130
14637992020,0cyclictest0-21swapper/007:38:140
14637992020,0cyclictest0-21swapper/007:23:130
1463799200,20cyclictest0-21swapper/010:33:110
1463799200,20cyclictest0-21swapper/009:28:160
1463799200,0cyclictest0-21swapper/009:13:150
14649991919,0cyclictest0-21swapper/311:23:133
14649991919,0cyclictest0-21swapper/310:48:143
14649991919,0cyclictest0-21swapper/310:03:153
14649991919,0cyclictest0-21swapper/307:18:133
14649991917,1cyclictest3851-21munin-run11:14:593
14644991919,0cyclictest0-21swapper/210:58:152
14644991919,0cyclictest0-21swapper/209:00:002
14644991919,0cyclictest0-21swapper/208:48:142
14644991919,0cyclictest0-21swapper/208:38:142
1464499190,1cyclictest313-21systemd-journal10:35:002
14642991919,0cyclictest0-21swapper/112:08:151
14642991919,0cyclictest0-21swapper/111:43:141
14642991918,1cyclictest0-21swapper/109:25:001
14642991917,1cyclictest27296-21apt-get07:38:131
14642991917,1cyclictest1417-21cron11:10:001
14637991919,0cyclictest0-21swapper/012:13:150
14637991919,0cyclictest0-21swapper/010:53:160
14637991919,0cyclictest0-21swapper/010:23:140
14637991919,0cyclictest0-21swapper/009:43:140
14637991917,1cyclictest25552-21munin-run12:00:000
14637991916,2cyclictest1588-21latency_hist07:50:000
14649991818,0cyclictest0-21swapper/311:18:143
14649991818,0cyclictest0-21swapper/311:08:163
14649991817,0cyclictest0-21swapper/311:53:173
14644991818,0cyclictest26050-21apt-get12:03:162
14644991818,0cyclictest18783-21apt-get11:48:162
14644991818,0cyclictest0-21swapper/211:38:152
14644991818,0cyclictest0-21swapper/209:13:152
14644991818,0cyclictest0-21swapper/207:43:122
14644991816,1cyclictest29225-21basename07:40:012
14637991818,0cyclictest0-21swapper/009:03:140
14637991818,0cyclictest0-21swapper/008:23:130
14637991816,1cyclictest19454-21awk07:20:000
14637991816,1cyclictest0-21swapper/008:08:160
14637991815,2cyclictest22498-21perf09:40:000
14637991814,1cyclictest5462-21cut11:18:200
1463799180,17cyclictest29057-21grep11:00:000
14649991717,0cyclictest0-21swapper/310:58:143
14649991717,0cyclictest0-21swapper/309:38:153
14649991716,1cyclictest0-21swapper/311:05:003
14644991717,0cyclictest19321-21apt-get08:28:122
14644991717,0cyclictest0-21swapper/209:18:132
14644991717,0cyclictest0-21swapper/209:08:122
14644991717,0cyclictest0-21swapper/208:23:132
14644991717,0cyclictest0-21swapper/207:48:152
14644991716,1cyclictest5691-21perf09:05:002
14644991714,2cyclictest26591-21apt08:43:132
14642991716,1cyclictest0-21swapper/112:05:001
14642991715,1cyclictest23625-21perf08:35:001
14642991715,1cyclictest0-21swapper/112:18:221
14637991717,0cyclictest0-21swapper/012:18:150
14637991716,0cyclictest0-21swapper/010:18:140
14637991715,1cyclictest0-21swapper/009:25:000
14649991616,0cyclictest12177-21perf10:25:003
14649991616,0cyclictest0-21swapper/312:23:133
14649991616,0cyclictest0-21swapper/311:33:143
14649991616,0cyclictest0-21swapper/309:13:153
14644991616,0cyclictest7861-21apt-get10:18:142
14644991616,0cyclictest0-21swapper/212:30:022
14644991616,0cyclictest0-21swapper/207:33:132
14644991614,1cyclictest5459-21apt-get10:13:132
14644991614,1cyclictest31457-21grep11:04:592
14644991613,2cyclictest32576-21head11:08:182
14644991613,2cyclictest313-21systemd-journal07:25:002
1464499160,0cyclictest0-21swapper/207:58:132
14642991616,0cyclictest22339-21apt-get07:28:131
14642991616,0cyclictest0-21swapper/111:18:141
14642991616,0cyclictest0-21swapper/110:28:141
14642991614,1cyclictest0-21swapper/111:55:001
1464299160,15cyclictest12932-21mailstats11:33:221
14637991616,0cyclictest0-21swapper/007:58:140
14637991616,0cyclictest0-21swapper/007:33:130
14637991614,1cyclictest3839-21sh11:15:000
1463799160,7cyclictest0-21swapper/011:23:130
14649991515,0cyclictest0-21swapper/311:48:173
14644991515,0cyclictest28444-21apt-get12:08:142
14644991515,0cyclictest0-21swapper/208:53:132
14642991515,0cyclictest0-21swapper/108:28:131
14642991514,0cyclictest0-21swapper/108:43:131
14637991515,0cyclictest0-21swapper/008:58:120
14649991414,0cyclictest0-21swapper/308:55:393
14649991414,0cyclictest0-21swapper/308:08:123
14649991414,0cyclictest0-21swapper/307:51:133
1464999140,14cyclictest0-21swapper/312:25:263
1464999140,14cyclictest0-21swapper/309:31:273
1464999140,14cyclictest0-21swapper/309:00:043
1464999140,14cyclictest0-21swapper/307:33:223
1464999140,14cyclictest0-21swapper/307:22:593
1464999140,0cyclictest0-21swapper/310:26:023
1464999140,0cyclictest0-21swapper/309:51:353
1464999140,0cyclictest0-21swapper/309:48:013
1464999140,0cyclictest0-21swapper/309:26:243
1464999140,0cyclictest0-21swapper/308:36:003
14644991414,0cyclictest0-21swapper/209:56:202
14644991414,0cyclictest0-21swapper/209:30:412
1464499140,14cyclictest0-21swapper/211:58:112
1464499140,14cyclictest0-21swapper/210:51:282
1464499140,14cyclictest0-21swapper/207:50:022
14642991414,0cyclictest0-21swapper/110:46:001
14642991414,0cyclictest0-21swapper/110:15:441
14642991414,0cyclictest0-21swapper/107:45:381
1464299140,14cyclictest0-21swapper/111:35:081
1464299140,14cyclictest0-21swapper/109:35:311
1464299140,14cyclictest0-21swapper/109:18:181
1464299140,14cyclictest0-21swapper/108:53:461
1464299140,14cyclictest0-21swapper/108:21:561
1464299140,14cyclictest0-21swapper/107:23:041
1464299140,0cyclictest0-21swapper/111:01:281
14637991414,0cyclictest0-21swapper/009:15:200
1463799140,14cyclictest0-21swapper/009:09:010
1463799140,0cyclictest0-21swapper/008:38:180
14649991313,0cyclictest0-21swapper/308:45:303
14644991313,0cyclictest0-21swapper/210:23:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional