You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 20:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Mon Nov 24, 2025 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1133999330,32cyclictest0-21swapper/110:22:221
11339992923,5cyclictest4507-21perf11:19:591
1134399240,23cyclictest0-21swapper/209:52:212
1133999240,0cyclictest0-21swapper/111:47:211
11333992422,2cyclictest13485-21perf08:19:590
11333992422,1cyclictest768-21runrttasks07:27:230
11351992322,1cyclictest6708-21apt-get08:07:213
11351992321,1cyclictest30143-21apt-get11:07:223
1135199230,22cyclictest0-21swapper/312:17:233
1134399230,22cyclictest3455-21awk08:00:002
11339992323,0cyclictest0-21swapper/110:17:231
1133999231,21cyclictest0-21swapper/111:07:211
1133999230,2cyclictest12451-21awk10:30:001
1133999230,22cyclictest4224-21apt-get12:27:231
1133999230,22cyclictest0-21swapper/110:00:001
1133399230,22cyclictest24485-21latency_hist10:55:000
1133399230,22cyclictest23775-21apt12:00:000
11351992220,1cyclictest1828-21apt-get12:22:223
1135199220,21cyclictest593-21dbus-daemon11:40:003
1135199220,21cyclictest30759-21munin-run08:55:003
1135199220,21cyclictest22108-21grep10:50:003
1135199220,21cyclictest0-21swapper/310:42:223
1135199220,21cyclictest0-21swapper/309:15:023
1135199220,21cyclictest0-21swapper/309:07:213
1135199220,21cyclictest0-21swapper/309:07:213
1134399220,21cyclictest8661-21latency_hist08:10:002
1134399220,21cyclictest21361-21idleruntime-cro11:55:002
1134399220,21cyclictest18283-21perf08:30:012
1134399220,21cyclictest16734-21apt-get07:20:022
1134399220,21cyclictest13559-21perf09:25:012
1134399220,1cyclictest15876-21latency_hist08:25:002
11339992220,1cyclictest593-21dbus-daemon07:40:011
11339992220,1cyclictest15960-21grep09:30:001
11339992216,5cyclictest20735-21apt-get08:35:021
1133999220,21cyclictest16104-21apt-get07:20:021
1133999220,21cyclictest0-21swapper/111:32:211
1133999220,21cyclictest0-21swapper/110:12:211
1133999220,21cyclictest0-21swapper/108:42:201
1133999220,21cyclictest0-21swapper/107:20:001
1133999220,1cyclictest11064-21apt08:15:001
11333992222,0cyclictest0-21swapper/008:22:210
11333992220,1cyclictest593-21dbus-daemon07:45:010
11333992220,1cyclictest12442-21wc10:30:000
11333992220,1cyclictest0-21swapper/007:55:000
11333992219,2cyclictest28350-21sh08:50:000
1133399220,21cyclictest25951-21latency_hist08:45:000
1133399220,21cyclictest12150-21apt-get11:37:220
1133399220,21cyclictest0-21swapper/007:12:230
1133399220,1cyclictest0-21swapper/007:17:230
1133399220,0cyclictest0-21swapper/010:57:200
11351992121,0cyclictest0-21swapper/311:57:213
11351992121,0cyclictest0-21swapper/308:02:213
11351992120,1cyclictest26717-21apt-get07:42:213
1135199210,21cyclictest0-21swapper/309:47:203
1135199210,20cyclictest5226-21grep10:15:013
1135199210,20cyclictest0-21swapper/309:42:213
1135199210,20cyclictest0-21swapper/309:12:243
1135199210,20cyclictest0-21swapper/308:37:223
1135199210,0cyclictest0-21swapper/308:17:203
11343992120,1cyclictest11737-21apt-get07:12:222
11343992120,0cyclictest29314-21apt-get11:05:022
1134399210,20cyclictest8782-21apt-get09:17:222
1134399210,20cyclictest0-21swapper/209:02:212
1134399210,20cyclictest0-21swapper/208:02:202
1134399210,1cyclictest28989-21perf12:10:002
11339992120,0cyclictest0-21swapper/107:42:211
11339992119,2cyclictest0-21swapper/110:47:191
11339992119,1cyclictest5560-21munin-run09:10:001
11339992119,1cyclictest5560-21munin-run09:10:001
11339992119,1cyclictest15895-21latency_hist08:25:001
11339992119,1cyclictest0-21swapper/109:20:011
11339992119,1cyclictest0-21swapper/109:05:001
11339992118,3cyclictest0-21swapper/109:37:221
11339992113,7cyclictest0-21swapper/111:22:211
1133999210,0cyclictest21264-21apt-get09:42:221
1133999210,0cyclictest11507-21apt-get08:17:201
1133999210,0cyclictest0-21swapper/108:07:211
11333992121,0cyclictest0-21swapper/010:02:190
1133399210,20cyclictest12874-21apt-get10:32:230
1133399210,20cyclictest0-21swapper/012:07:210
1133399210,0cyclictest0-21swapper/010:12:220
1133399210,0cyclictest0-21swapper/009:52:210
11351992020,0cyclictest8069-21apt-get10:22:223
11351992020,0cyclictest0-21swapper/312:12:223
1135199200,20cyclictest0-21swapper/310:32:223
1135199200,0cyclictest23988-21apt-get08:42:203
1135199200,0cyclictest0-21swapper/312:37:213
1135199200,0cyclictest0-21swapper/311:17:223
1135199200,0cyclictest0-21swapper/309:37:223
1135199200,0cyclictest0-21swapper/307:57:213
1135199200,0cyclictest0-21swapper/307:12:223
11343992020,0cyclictest9018-21apt-get12:37:222
11343992020,0cyclictest6620-21apt-get12:32:202
11343992020,0cyclictest28455-21apt-get09:57:232
11343992020,0cyclictest20137-21apt-get10:47:192
11343992020,0cyclictest0-21swapper/212:27:232
11343992020,0cyclictest0-21swapper/211:32:222
11343992020,0cyclictest0-21swapper/211:22:212
11343992020,0cyclictest0-21swapper/211:12:222
11343992020,0cyclictest0-21swapper/209:27:202
11343992019,1cyclictest23232-21perf09:45:002
11343992019,1cyclictest18793-21apt-get09:37:222
11343992018,1cyclictest0-21swapper/211:20:002
11343992018,1cyclictest0-21swapper/209:50:002
1134399200,0cyclictest0-21swapper/212:22:222
1134399200,0cyclictest0-21swapper/208:42:192
1134399200,0cyclictest0-21swapper/208:32:212
1134399200,0cyclictest0-21swapper/207:47:212
11339992020,0cyclictest0-21swapper/112:22:221
11339992020,0cyclictest0-21swapper/111:57:221
11339992020,0cyclictest0-21swapper/111:42:231
11339992020,0cyclictest0-21swapper/111:37:231
11339992020,0cyclictest0-21swapper/111:27:211
11339992020,0cyclictest0-21swapper/110:32:231
11339992020,0cyclictest0-21swapper/109:52:211
11339992020,0cyclictest0-21swapper/108:57:211
1133999200,20cyclictest0-21swapper/108:47:211
1133999200,19cyclictest32539-21apt-get11:12:211
11333992020,0cyclictest0-21swapper/012:22:210
11333992020,0cyclictest0-21swapper/012:02:210
11333992020,0cyclictest0-21swapper/011:22:210
11333992020,0cyclictest0-21swapper/010:42:230
11333992020,0cyclictest0-21swapper/007:37:200
11333992019,0cyclictest0-21swapper/011:42:230
1133399200,20cyclictest0-21swapper/011:12:220
1133399200,20cyclictest0-21swapper/010:07:230
1133399200,20cyclictest0-21swapper/008:07:210
1133399200,0cyclictest0-21swapper/012:37:210
1133399200,0cyclictest0-21swapper/007:47:200
11351991919,0cyclictest0-21swapper/311:22:213
11351991919,0cyclictest0-21swapper/310:17:233
11351991919,0cyclictest0-21swapper/307:47:213
11351991919,0cyclictest0-21swapper/307:37:203
11351991917,1cyclictest16095-21cron07:20:003
11343991919,0cyclictest24230-21apt-get12:02:212
11343991919,0cyclictest22535-21apt-get10:52:212
11343991919,0cyclictest21793-21apt-get07:32:192
11343991919,0cyclictest10466-21apt-get10:27:202
11343991919,0cyclictest0-21swapper/212:17:232
11343991919,0cyclictest0-21swapper/210:02:202
11343991919,0cyclictest0-21swapper/207:52:192
11343991919,0cyclictest0-21swapper/207:27:222
11339991919,0cyclictest846-21apt-get10:07:221
11339991919,0cyclictest19349-21apt-get07:27:221
11339991919,0cyclictest1485-21apt-get07:57:221
11339991919,0cyclictest0-21swapper/112:07:211
11339991919,0cyclictest0-21swapper/112:02:211
11339991919,0cyclictest0-21swapper/110:52:211
11339991919,0cyclictest0-21swapper/109:47:211
11339991918,1cyclictest17691-21apt-get10:42:231
11333991919,0cyclictest8782-21apt-get09:17:220
11333991919,0cyclictest0-21swapper/011:52:220
11333991919,0cyclictest0-21swapper/011:17:220
11333991919,0cyclictest0-21swapper/011:07:220
11333991919,0cyclictest0-21swapper/010:47:180
11333991919,0cyclictest0-21swapper/010:37:230
11333991919,0cyclictest0-21swapper/007:57:220
11333991918,0cyclictest0-21swapper/010:25:000
1133399190,0cyclictest0-21swapper/012:12:210
1133399190,0cyclictest0-21swapper/008:32:210
11351991818,0cyclictest30868-21apt-get10:02:203
11351991818,0cyclictest16391-21apt-get09:32:203
11351991818,0cyclictest0-21swapper/312:07:213
11351991818,0cyclictest0-21swapper/309:57:233
11351991817,1cyclictest31182-21apt-get08:57:203
11351991817,0cyclictest0-21swapper/311:52:223
11343991817,1cyclictest26388-21apt-get08:47:202
11339991818,0cyclictest24934-21apt-get10:57:211
11339991818,0cyclictest18723-21apt-get08:32:201
11339991818,0cyclictest0-21swapper/112:37:201
11339991818,0cyclictest0-21swapper/107:32:181
11339991817,0cyclictest29111-21apt-get07:47:211
1133999180,6cyclictest13458-21runrttasks07:12:321
11333991818,0cyclictest0-21swapper/012:32:200
11333991818,0cyclictest0-21swapper/012:17:210
11333991818,0cyclictest0-21swapper/011:27:210
11333991818,0cyclictest0-21swapper/011:05:000
11333991818,0cyclictest0-21swapper/009:47:210
11333991818,0cyclictest0-21swapper/009:12:230
11333991818,0cyclictest0-21swapper/007:20:010
11333991817,0cyclictest3915-21apt-get08:02:210
11351991717,0cyclictest0-21swapper/311:27:203
11351991717,0cyclictest0-21swapper/309:27:203
11351991717,0cyclictest0-21swapper/307:32:203
11351991716,0cyclictest0-21swapper/308:22:213
11351991715,1cyclictest3150-21latency_hist09:04:593
11343991717,0cyclictest7343-21apt-get11:27:212
11343991717,0cyclictest5655-21apt-get10:17:232
11343991717,0cyclictest0-21swapper/210:32:232
11343991717,0cyclictest0-21swapper/210:07:222
1134399170,14cyclictest4389-21irqstats09:07:252
1134399170,14cyclictest4389-21irqstats09:07:252
1133999170,14cyclictest0-21swapper/111:02:311
11333991717,0cyclictest13994-21apt-get09:27:200
11333991717,0cyclictest0-21swapper/011:32:220
11333991717,0cyclictest0-21swapper/009:42:210
11333991717,0cyclictest0-21swapper/009:02:210
11351991616,0cyclictest16324-21apt-get08:27:213
11351991616,0cyclictest0-21swapper/312:02:213
11351991616,0cyclictest0-21swapper/311:02:223
11351991616,0cyclictest0-21swapper/310:52:213
11351991616,0cyclictest0-21swapper/310:27:213
11351991616,0cyclictest0-21swapper/307:22:203
11351991614,1cyclictest16538-21perf11:45:013
11351991614,1cyclictest0-21swapper/311:32:233
1135199160,16cyclictest0-21swapper/312:32:203
1135199160,15cyclictest26234-21cut09:52:223
1135199160,15cyclictest0-21swapper/311:50:013
11343991616,0cyclictest14571-21apt-get11:42:222
11343991616,0cyclictest0-21swapper/212:12:222
11343991616,0cyclictest0-21swapper/210:42:242
11343991616,0cyclictest0-21swapper/210:12:222
11343991616,0cyclictest0-21swapper/208:12:212
11343991615,1cyclictest14137-21apt-get07:17:222
11339991616,0cyclictest0-21swapper/109:35:001
11339991616,0cyclictest0-21swapper/109:12:241
11339991615,0cyclictest0-21swapper/109:17:211
11339991614,2cyclictest0-21swapper/107:55:001
11333991616,0cyclictest0-21swapper/009:32:200
11333991616,0cyclictest0-21swapper/008:52:220
11333991616,0cyclictest0-21swapper/008:27:210
11333991616,0cyclictest0-21swapper/008:12:200
11333991612,3cyclictest313-21systemd-journal09:10:000
11333991612,3cyclictest313-21systemd-journal09:10:000
11351991515,0cyclictest15272-21apt-get10:37:233
11351991514,1cyclictest0-21swapper/310:57:213
11351991511,4cyclictest0-21swapper/307:27:113
1135199150,1cyclictest1271-21diskstats10:07:243
11343991515,0cyclictest0-21swapper/210:22:212
11343991515,0cyclictest0-21swapper/208:57:212
11343991513,1cyclictest17490-21perl11:47:222
1134399150,6cyclictest0-21swapper/210:57:212
1134399150,1cyclictest13199-21ntp_states11:37:272
11339991515,0cyclictest0-21swapper/112:12:351
11339991515,0cyclictest0-21swapper/108:02:211
11339991514,1cyclictest0-21swapper/110:36:511
11339991514,0cyclictest0-21swapper/110:02:201
1133999150,15cyclictest16832-21tune2fs08:27:221
1133999150,14cyclictest0-21swapper/112:32:281
1133399150,1cyclictest0-21swapper/011:47:350
1133399150,12cyclictest1686-21apache209:35:210
1133399150,0cyclictest0-21swapper/012:27:230
11351991414,0cyclictest0-21swapper/307:52:443
1135199140,14cyclictest0-21swapper/312:27:323
1135199140,14cyclictest0-21swapper/308:46:133
1135199140,14cyclictest0-21swapper/308:11:223
1135199140,0cyclictest0-21swapper/311:11:043
1135199140,0cyclictest0-21swapper/308:32:203
11343991414,0cyclictest0-21swapper/211:02:212
11343991414,0cyclictest0-21swapper/209:30:142
1134399140,14cyclictest0-21swapper/211:55:332
1134399140,14cyclictest0-21swapper/210:35:362
1134399140,14cyclictest0-21swapper/209:11:132
1134399140,14cyclictest0-21swapper/208:36:432
1134399140,14cyclictest0-21swapper/207:40:462
1134399140,14cyclictest0-21swapper/207:37:322
1134399140,0cyclictest0-21swapper/208:52:312
1134399140,0cyclictest0-21swapper/208:18:012
11339991414,0cyclictest0-21swapper/112:15:471
11339991414,0cyclictest0-21swapper/108:52:551
1133999140,14cyclictest0-21swapper/111:52:141
11333991414,0cyclictest0-21swapper/008:56:290
11333991414,0cyclictest0-21swapper/008:35:020
1133399140,1cyclictest0-21swapper/009:20:010
1133399140,14cyclictest0-21swapper/009:55:260
1133399140,14cyclictest0-21swapper/007:34:340
1133399140,0cyclictest0-21swapper/010:15:370
11351991313,0cyclictest0-21swapper/309:22:213
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional