You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 16:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8s.osadl.org (updated Sun Nov 16, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22438789912830,1279cyclictest24043302sleep309:14:563
22438789911700,1166cyclictest22835642sleep307:39:573
22438789911330,1129cyclictest23713622sleep308:49:573
22438789911130,1109cyclictest26718052sleep312:00:013
22438789910601,1055cyclictest2728105-21sleep12:35:003
2243878994170,414cyclictest2452963-21turbostat09:49:553
2243878992630,260cyclictest2558195-21turbostat10:54:553
24368472230,2sleep30-21swapper/309:35:003
2243878991916,1cyclictest0-21swapper/310:10:143
2243878991815,1cyclictest0-21swapper/307:20:223
224387899176,8cyclictest2412437-21turbostat09:24:553
2243878991714,1cyclictest0-21swapper/310:40:213
2243878991714,1cyclictest0-21swapper/309:45:013
2243878991714,1cyclictest0-21swapper/307:30:253
224387899170,13cyclictest2622922-21turbostat11:30:003
224387899166,7cyclictest25790302sleep311:00:203
2243878991614,0cyclictest0-21swapper/311:37:333
2243878991614,0cyclictest0-21swapper/311:37:333
224387899161,11cyclictest2244071-21taskset07:10:123
26879792151,4sleep12688004-21apt-get12:10:141
26715272150,1sleep12671558-21munin-run12:00:001
224387899156,5cyclictest2333608-21turbostat08:20:003
224387899154,7cyclictest25178372sleep310:25:133
2243878991513,1cyclictest0-21swapper/311:40:203
2243878991513,1cyclictest0-21swapper/308:25:003
2243878991513,1cyclictest0-21swapper/308:10:013
2243878991512,1cyclictest0-21swapper/310:57:483
224387899151,11cyclictest2704074-21taskset12:20:013
224387899151,11cyclictest2461148-21taskset09:50:023
224387899151,11cyclictest2421016-21taskset09:25:033
224387899151,11cyclictest2352853-21chrt08:35:023
224387899151,10cyclictest2477020-21turbostat10:00:003
27112272141,4sleep22711235-21ssh12:23:092
224387899141,9cyclictest24856812sleep310:05:143
2243878991412,1cyclictest0-21swapper/312:11:203
2243878991412,1cyclictest0-21swapper/311:57:103
2243878991412,1cyclictest0-21swapper/310:15:533
2243878991412,1cyclictest0-21swapper/308:53:373
2243878991412,1cyclictest0-21swapper/308:34:003
2243878991412,1cyclictest0-21swapper/307:48:443
2243878991412,1cyclictest0-21swapper/307:28:503
224387899141,11cyclictest2444890-21turbostat09:40:003
2243878991411,1cyclictest0-21swapper/311:54:193
2243878991411,1cyclictest0-21swapper/309:07:053
224387899141,10cyclictest2680018-21taskset12:05:143
224387899141,10cyclictest2631189-21taskset11:34:533
224387899141,10cyclictest2252565-21chrt07:15:163
190898140,9rtkit-daemon2268236-21cut07:25:182
190898140,2rtkit-daemon0-21swapper/207:17:242
24291602130,2sleep00-21swapper/009:30:130
23092582131,5sleep12309771-21cut08:00:191
224387899134,7cyclictest2516731-21taskset10:23:083
224387899131,9cyclictest2728464-21taskset12:35:153
224387899131,9cyclictest2591063-21taskset11:10:123
224387899131,9cyclictest2555992-21taskset10:45:243
224387899131,9cyclictest2537257-21taskset10:35:183
224387899131,9cyclictest2483745-21chrt10:01:203
224387899131,9cyclictest2308854-21chrt08:00:123
224387899131,9cyclictest2302793-21taskset07:55:153
224387899131,9cyclictest2296652-21chrt07:50:143
2243878991312,0cyclictest0-21swapper/311:18:443
2243878991312,0cyclictest0-21swapper/308:41:403
2243878991311,1cyclictest0-21swapper/311:05:153
224387899131,10cyclictest2711356-21taskset12:23:253
224387899130,10cyclictest2652682-21taskset11:45:193
26854052121,4sleep12685684-21grep12:05:251
224387899124,6cyclictest27126052sleep312:25:193
224387899121,9cyclictest2610933-21taskset11:20:183
224387899121,8cyclictest2404666-21taskset09:15:083
224387899121,8cyclictest2377911-21taskset08:55:133
2243878991212,0cyclictest0-21swapper/308:25:423
224387899120,9cyclictest2525948-21taskset10:30:053
224387899120,9cyclictest2384139-21taskset09:00:023
224387899120,10cyclictest2289584-21chrt07:41:253
190898120,8rtkit-daemon1928-21sendmail-mta11:58:310
24439022110,8sleep22443906-21ssh09:37:442
24271592110,4sleep20-21swapper/209:26:102
224387899111,8cyclictest2327032-21taskset08:11:103
190898110,4rtkit-daemon667-21snmpd09:54:121
190898110,4rtkit-daemon2613831-21ssh11:22:320
190898110,4rtkit-daemon2402852-21ssh09:11:210
190898110,1rtkit-daemon0-21swapper/012:08:350
26950702101,6sleep20-21swapper/212:13:362
26132042100,7sleep12613212-21ssh11:21:051
25665062100,6sleep22566540-21apt-get10:55:142
25498102100,1sleep10-21swapper/110:44:191
24583532100,7sleep22458416-21awk09:45:212
22711842102,4sleep12271217-21apt-get07:30:141
190898100,6rtkit-daemon667-21snmpd11:01:210
190898100,6rtkit-daemon2616746-21tr11:25:200
190898100,6rtkit-daemon2339695-21chrt08:22:320
190898100,5rtkit-daemon2290537-21sort07:45:170
190898100,2rtkit-daemon2376789-21awk08:50:210
190898100,2rtkit-daemon0-21swapper/207:07:232
190898100,1rtkit-daemon0-21swapper/207:33:462
2702548291,6sleep22702550-21ssh12:17:202
2649180290,6sleep12650904-21cut11:45:181
2645092291,5sleep22645341-21grep11:40:262
2571579291,6sleep12571723-21/usr/sbin/munin10:55:211
2542856290,7sleep02543417-21irqrtprio10:40:170
2516585291,2sleep10-21swapper/110:22:511
2493866291,5sleep02494072-21/usr/sbin/munin10:10:160
2414274290,6sleep02416066-21cut09:20:190
2404356291,5sleep22404374-21munin-run09:15:002
2309772291,5sleep20-21swapper/208:00:202
2244035291,4sleep22244068-21apt-get07:10:122
19089890,5rtkit-daemon667-21snmpd07:54:340
19089890,5rtkit-daemon2689712-21kthreadcore12:10:180
19089890,5rtkit-daemon2647915-21diskstats11:45:140
19089890,5rtkit-daemon2512869-21kthreadcore10:20:190
19089890,5rtkit-daemon2477525-21echo10:00:011
19089890,5rtkit-daemon2359543-21hddtemp_smartct08:40:180
19089890,5rtkit-daemon1928-21sendmail-mta11:52:310
19089890,4rtkit-daemon299-21systemd-journal12:32:512
19089890,4rtkit-daemon2729961-21ssh12:35:200
19089890,4rtkit-daemon2711253-21ssh12:23:111
19089890,4rtkit-daemon2678208-21ssh12:01:420
19089890,3rtkit-daemon2638055-21ssh11:37:070
19089890,3rtkit-daemon2638055-21ssh11:37:070
19089890,3rtkit-daemon2606683-21ssh11:19:450
19089890,3rtkit-daemon2419081-21ssh09:21:382
2734842280,1sleep20-21swapper/212:37:002
2679978281,3sleep22680013-21apt-get12:05:142
2584458282,3sleep12586057-21kthreadcore11:05:191
2557011280,1sleep00-21swapper/010:47:180
2534000281,4sleep12534032-21ssh10:35:141
2523109281,4sleep02523173-21ntp_states10:25:230
2475796280,6sleep02475803-21sh09:57:140
2475274281,4sleep12475282-21ssh09:55:581
2453231282,3sleep12453266-21apt-get09:45:121
2341598280,6sleep12343029-21cut08:25:191
2327587280,2sleep00-21swapper/008:15:120
2294396280,5sleep12295116-21wc07:45:201
2264905281,3sleep12264938-21apt-get07:25:131
2237766280,5sleep30-21swapper/307:05:123
19089880,5rtkit-daemon671-21ntpd12:15:261
19089880,5rtkit-daemon436-21systemd-logind10:31:370
19089880,4rtkit-daemon81364-21packagekitd07:40:200
19089880,4rtkit-daemon81364-21packagekitd07:36:102
19089880,4rtkit-daemon674-21lldpd11:07:450
19089880,4rtkit-daemon667-21snmpd07:21:092
19089880,4rtkit-daemon473-21gmain11:44:300
19089880,4rtkit-daemon436-21systemd-logind10:53:370
19089880,4rtkit-daemon407-21rpcbind08:59:270
19089880,4rtkit-daemon2594806-21kthreadcore11:10:180
19089880,4rtkit-daemon2425914-21kthreadcore09:25:220
19089880,4rtkit-daemon2308892-21apt-get08:00:150
19089880,4rtkit-daemon2250230-21munin-node10:14:011
19089880,3rtkit-daemon2539771-21ssh10:35:270
19089880,3rtkit-daemon2321370-21apt-get08:10:140
19089880,1rtkit-daemon0-21swapper/209:15:022
19089880,1rtkit-daemon0-21swapper/008:06:370
2720441270,3sleep12720474-21apt-get12:30:151
2718973270,5sleep02718981-21ssh12:27:280
2703428270,1sleep00-21swapper/012:19:190
2677512270,4sleep22677683-21vmstat12:00:252
2661036270,4sleep12661077-21ssh11:50:231
2637761270,1sleep20-21swapper/211:36:252
2637761270,1sleep20-21swapper/211:36:252
2631689271,3sleep12631808-21ssh11:35:161
2631689271,3sleep12631808-21ssh11:35:161
2605607270,2sleep10-21swapper/111:17:181
2591431270,4sleep22591927-21sed11:10:162
2535480270,5sleep22537256-21kthreadcore10:35:182
2527159270,4sleep12528230-21tr10:30:201
2520606270,4sleep12522758-21ssh10:25:191
2478117270,4sleep02479116-21irqstats10:00:180
2452261270,4sleep02452269-21ssh09:43:270
2390603270,4sleep22390803-21hddtemp_smartct09:05:182
2369131270,4sleep22370330-21memory08:45:192
2359450270,1sleep20-21swapper/208:40:182
2283886270,1sleep20-21swapper/207:40:172
2283850270,3sleep12283883-21apt-get07:40:161
2270626270,1sleep00-21swapper/007:26:120
2263922270,3sleep02264031-21find07:20:230
19089870,4rtkit-daemon671-21ntpd11:30:230
19089870,4rtkit-daemon473-21gmain09:08:340
19089870,4rtkit-daemon473-21gmain07:56:380
19089870,4rtkit-daemon421-21avahi-daemon10:56:090
19089870,4rtkit-daemon421-21avahi-daemon10:05:081
19089870,4rtkit-daemon2250230-21munin-node10:11:082
19089870,4rtkit-daemon0-21swapper/009:35:160
19089870,3rtkit-daemon667-21snmpd09:40:191
19089870,3rtkit-daemon667-21snmpd09:37:131
19089870,3rtkit-daemon667-21snmpd09:00:010
19089870,3rtkit-daemon667-21snmpd08:28:500
19089870,3rtkit-daemon473-21gmain10:17:300
19089870,3rtkit-daemon2712438-21apt-get12:25:161
19089870,3rtkit-daemon2506854-21sh10:15:221
19089870,3rtkit-daemon2492936-21pool-udisksd10:09:192
19089870,3rtkit-daemon2356841-21kthreadcore08:35:200
19089870,3rtkit-daemon2250230-21munin-node08:31:320
2725286260,3sleep02725545-21needreboot12:30:210
2719659261,2sleep20-21swapper/212:29:062
2679187260,4sleep10-21swapper/112:04:011
2659091261,2sleep20-21swapper/211:50:192
2630565260,1sleep10-21swapper/111:33:281
2621453260,3sleep12621461-21ssh11:26:341
2615231260,4sleep22615270-21apt-get11:25:152
2608888260,4sleep22610930-21kthreadcore11:20:182
2597709260,1sleep10-21swapper/111:12:361
2588804260,3sleep22588812-21ssh11:05:362
2575901260,4sleep12576920-21kthreadcore11:00:181
2565105260,1sleep20-21swapper/210:52:332
2557597260,1sleep20-21swapper/210:48:412
2553247260,1sleep10-21swapper/110:45:191
2544814260,3sleep22546839-21ssh10:40:192
2524811260,4sleep22524819-21ssh10:28:052
2501682260,2sleep20-21swapper/210:15:152
2487169261,2sleep02488716-21grep10:05:190
2482876261,3sleep2421-21avahi-daemon10:00:242
2476349260,1sleep20-21swapper/209:58:322
2461224260,1sleep00-21swapper/009:50:120
2459397260,1sleep00-21swapper/009:46:280
2445222261,2sleep20-21swapper/209:40:132
2427543260,4sleep12427551-21ssh09:27:091
2412982261,3sleep10-21swapper/109:20:161
2389467260,3sleep12389719-21sed09:00:251
2352867260,3sleep22352900-21apt-get08:35:142
2348257260,3sleep12350363-21kthreadcore08:30:201
2316562260,4sleep22317836-21kthreadcore08:05:212
2315109260,3sleep12315142-21apt-get08:05:161
2304061261,3sleep20-21swapper/207:55:192
2301531260,3sleep22301656-21python307:50:222
2297828260,3sleep12299152-21cut07:50:171
2252376260,3sleep12252409-21apt-get07:15:151
2240055260,3sleep12242242-21cut07:05:181
19089860,3rtkit-daemon1928-21sendmail-mta08:46:160
2734237251,2sleep10-21swapper/112:35:341
2671199250,1sleep20-21swapper/211:59:092
2647914250,3sleep20-21swapper/211:45:152
2645442250,2sleep10-21swapper/111:40:291
2564407250,2sleep12564415-21ssh10:50:561
2532641250,1sleep20-21swapper/210:32:342
2430943250,2sleep12433002-21tr09:30:181
2410848250,1sleep10-21swapper/109:16:151
2396726250,1sleep10-21swapper/109:10:101
2384153250,2sleep20-21swapper/209:00:142
2383565250,1sleep20-21swapper/208:56:052
2377272250,1sleep20-21swapper/208:50:322
2352010250,1sleep20-21swapper/208:30:412
2339885250,3sleep22339921-21systemd-run08:25:002
2321749251,2sleep20-21swapper/208:10:172
2302604250,2sleep10-21swapper/107:55:141
2277414250,1sleep10-21swapper/107:35:151
2263613250,1sleep10-21swapper/107:20:201
2243797250,3sleep00-21swapper/007:10:020
2238860250,2sleep00-21swapper/007:05:170
2627436240,2sleep22628538-21ssh11:30:202
2599494240,2sleep20-21swapper/211:15:172
2580145240,2sleep20-21swapper/211:00:232
2515084240,2sleep22515208-21smartctl10:20:242
2462891240,2sleep20-21swapper/209:50:182
2435571240,2sleep22435579-21ssh09:32:032
2406992240,2sleep02407816-21kthreadcore09:15:180
2371818240,2sleep10-21swapper/108:50:151
2364798240,3sleep10-21swapper/108:40:571
2358766240,3sleep10-21swapper/108:38:491
2345975240,2sleep20-21swapper/208:28:322
2335310240,2sleep10-21swapper/108:20:181
2330037240,2sleep20-21swapper/208:15:182
2328865240,2sleep10-21swapper/108:15:171
2290121240,1sleep20-21swapper/207:45:132
2277605240,1sleep00-21swapper/007:35:180
2249810240,1sleep10-21swapper/107:11:471
2704522230,1sleep00-21swapper/012:20:150
2390446230,1sleep10-21swapper/109:05:171
2383743230,2sleep10-21swapper/108:58:241
2365406230,1sleep10-21swapper/108:45:141
2325848230,1sleep10-21swapper/108:10:201
2271384230,1sleep00-21swapper/007:30:170
2251038230,1sleep00-21swapper/007:15:030
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional