You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-26 - 10:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8s.osadl.org (updated Wed Nov 26, 2025 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21601149912320,1228cyclictest25970662sleep300:04:563
21601149911510,1147cyclictest22315882sleep320:05:003
21601149911241,1119cyclictest25325612sleep323:24:573
21601149910990,1094cyclictest2345894-21sleep21:30:013
21601149910361,1031cyclictest2410908-21sleep22:10:003
2160114992840,281cyclictest2467827-21turbostat22:49:553
2160114992115,3cyclictest2588974-21sleep00:00:003
216011499181,13cyclictest2565248-21taskset23:45:123
2160114991714,1cyclictest0-21swapper/321:20:193
216011499171,12cyclictest2492100-21turbostat23:00:003
216011499171,12cyclictest2386705-21taskset21:55:003
216011499171,12cyclictest2300733-21turbostat21:00:013
216011499171,12cyclictest2288161-21taskset20:50:013
216011499165,8cyclictest2307282-21turbostat21:05:013
2160114991613,1cyclictest0-21swapper/300:37:533
2160114991613,1cyclictest0-21swapper/300:05:163
216011499161,11cyclictest2269294-21turbostat20:35:003
24833512150,3sleep02483358-21sh22:53:320
216011499158,4cyclictest2448689-21taskset22:30:193
2160114991513,0cyclictest0-21swapper/323:19:453
2160114991512,1cyclictest0-21swapper/323:00:193
2160114991512,1cyclictest0-21swapper/321:55:143
2160114991512,1cyclictest0-21swapper/321:35:233
2160114991512,1cyclictest0-21swapper/320:10:523
216011499151,11cyclictest2532967-21taskset23:25:133
216011499151,11cyclictest2194158-21taskset19:35:163
216011499151,10cyclictest24516672sleep322:35:133
190898150,10rtkit-daemon2384888-21ssh21:51:060
216011499147,4cyclictest2612752-21taskset00:13:493
216011499145,6cyclictest26360002sleep300:26:293
216011499145,6cyclictest25628602sleep323:40:233
216011499145,6cyclictest23684352sleep321:40:263
216011499144,7cyclictest2467047-21taskset22:43:173
216011499143,8cyclictest2269519-21taskset20:35:013
216011499141,9cyclictest22253262sleep320:00:133
2160114991412,1cyclictest0-21swapper/323:52:113
2160114991412,1cyclictest0-21swapper/321:30:103
2160114991412,1cyclictest0-21swapper/320:20:593
2160114991412,1cyclictest0-21swapper/320:16:343
2160114991412,1cyclictest0-21swapper/320:16:343
2160114991411,1cyclictest0-21swapper/323:10:013
2160114991411,1cyclictest0-21swapper/319:18:253
2160114991411,1cyclictest0-21swapper/300:20:153
216011499141,10cyclictest2619520-21taskset00:15:463
216011499141,10cyclictest2514607-21chrt23:10:463
216011499141,10cyclictest2442849-21taskset22:29:143
216011499141,10cyclictest2320041-21taskset21:11:243
216011499141,10cyclictest2307514-21taskset21:05:073
216011499141,10cyclictest2276310-21taskset20:40:153
216011499141,10cyclictest2219037-21taskset19:55:123
25105172132,9sleep22512498-21cut23:10:182
216011499134,7cyclictest2166067-21taskset19:11:323
216011499131,9cyclictest2555232-21chrt23:36:153
216011499131,9cyclictest2419244-21chrt22:15:083
216011499131,9cyclictest2370712-21taskset21:45:063
216011499131,9cyclictest2294065-21taskset20:50:413
216011499131,9cyclictest2257905-21taskset20:25:173
216011499131,9cyclictest2257905-21taskset20:25:173
216011499131,9cyclictest2187007-21taskset19:25:393
2160114991311,1cyclictest0-21swapper/323:32:183
2160114991311,1cyclictest0-21swapper/321:17:563
2160114991310,1cyclictest0-21swapper/322:20:243
216011499130,10cyclictest2638019-21chrt00:30:123
216011499130,10cyclictest2411130-21taskset22:10:033
216011499130,10cyclictest2394980-21taskset22:00:013
216011499130,10cyclictest2218093-21chrt19:50:233
216011499130,10cyclictest2200160-21taskset19:40:013
190898130,8rtkit-daemon2369852-21ssh21:43:380
190898130,4rtkit-daemon2237557-21pool-udisksd20:09:090
190898130,1rtkit-daemon0-21swapper/021:27:430
25970902121,8sleep22597115-21apt00:05:002
25258402120,5sleep12526786-21grep23:20:191
24757782121,8sleep22475776-21ssh22:49:422
22683782120,8sleep22268434-21ntp_states20:30:212
216011499121,9cyclictest2482126-21taskset22:50:423
216011499121,8cyclictest2206436-21chrt19:45:023
216011499121,8cyclictest2192907-21taskset19:30:233
216011499120,9cyclictest2175026-21taskset19:20:103
23771832110,2sleep02377191-21ssh21:47:000
23012452111,5sleep12301278-21apt-get21:00:131
21539972111,5sleep02154030-21apt-get19:05:130
190898110,8rtkit-daemon2336447-21ssh21:21:490
190898110,6rtkit-daemon2616333-21cut00:15:170
190898110,4rtkit-daemon2596617-21ssh00:03:580
190898110,4rtkit-daemon2166508-21munin-node21:33:130
25903542101,7sleep12591229-21kthreadcore00:00:221
25071072100,1sleep20-21swapper/223:07:122
23128262100,3sleep12313010-21cat21:05:221
22450272100,7sleep12245970-21kthreadcore20:15:181
22450272100,7sleep12245970-21kthreadcore20:15:181
21599572102,3sleep30-21swapper/319:10:013
190898100,6rtkit-daemon2394372-21ssh21:59:180
190898100,5rtkit-daemon2530689-21ssh23:20:340
190898100,5rtkit-daemon2307551-21apt-get21:05:110
190898100,3rtkit-daemon2571359-21ssh23:46:160
2651360290,7sleep02651618-21sed00:35:310
2611343290,3sleep22611351-21ssh00:10:292
2572901291,5sleep12573031-21date23:50:001
2532923290,2sleep22532964-21ssh23:25:132
2352304291,5sleep12352312-21ssh21:31:021
2338129291,4sleep22338168-21apt-get21:25:112
2181330291,4sleep22181363-21apt-get19:25:142
19089890,5rtkit-daemon438-21wpa_supplicant22:09:090
19089890,5rtkit-daemon319-21systemd-udevd00:13:360
19089890,5rtkit-daemon2629454-21ssh00:24:520
19089890,5rtkit-daemon2539146-21ssh23:26:300
19089890,5rtkit-daemon2466714-21ssh22:42:280
19089890,5rtkit-daemon2212722-21runrttasks19:50:090
19089890,4rtkit-daemon2498512-21chrt23:00:570
19089890,4rtkit-daemon2473211-21memory22:45:180
19089890,4rtkit-daemon2458253-21ssh22:37:120
19089890,4rtkit-daemon2360932-21ssh21:36:440
19089890,4rtkit-daemon2320351-21ssh21:12:030
19089890,3rtkit-daemon473-21gmain00:06:220
19089890,3rtkit-daemon2434878-21ssh22:24:280
2646310281,5sleep22647248-21cut00:35:172
2644000280,1sleep20-21swapper/200:31:292
2628841282,1sleep2171rcu_preempt00:23:262
2612542280,5sleep12612550-21ssh00:13:161
2587512280,1sleep00-21swapper/023:56:360
2579374281,4sleep12579382-21ssh23:51:241
2555993280,6sleep22555995-21ssh23:38:072
2546769281,5sleep12546777-21ssh23:30:291
2522792281,4sleep22522800-21ssh23:16:072
2469630281,5sleep12471408-21kthreadcore22:45:161
2457626280,4sleep12457630-21ssh22:35:471
2402825280,6sleep0423-21dbus-daemon22:05:000
2367867281,4sleep22367931-21ntp_states21:40:222
2355073280,6sleep12355769-21sed21:35:201
2321930280,1sleep20-21swapper/221:15:082
2307194285,2sleep2431ktimers/221:04:072
2269932280,6sleep12270414-21cut20:35:171
2254921280,5sleep12255715-21grep20:20:191
2252837280,5sleep02254918-21kthreadcore20:20:180
2195311280,5sleep12196942-21grep19:35:191
2193852281,3sleep02193885-21apt-get19:35:140
2188992281,4sleep12190117-21grep19:30:181
19089880,5rtkit-daemon1928-21sendmail-mta19:42:570
19089880,4rtkit-daemon667-21snmpd23:32:090
19089880,4rtkit-daemon667-21snmpd23:17:580
19089880,4rtkit-daemon667-21snmpd20:01:100
19089880,4rtkit-daemon438-21wpa_supplicant21:01:250
19089880,4rtkit-daemon438-21wpa_supplicant20:38:440
19089880,4rtkit-daemon438-21wpa_supplicant20:27:440
19089880,4rtkit-daemon438-21wpa_supplicant20:27:430
19089880,4rtkit-daemon2638233-21hddtemp_smartct00:30:140
19089880,4rtkit-daemon2579905-21ssh23:52:400
19089880,4rtkit-daemon2514588-21ssh23:10:440
19089880,4rtkit-daemon2487420-21kthreadcore22:55:170
19089880,4rtkit-daemon2289853-21kthreadcore20:50:180
19089880,4rtkit-daemon2193029-21sed19:30:230
19089880,4rtkit-daemon2175218-21df_inode19:20:130
19089880,4rtkit-daemon2160624-21gltestperf19:10:160
19089880,4rtkit-daemon1223-21cron22:12:000
19089880,1rtkit-daemon0-21swapper/022:29:400
19089880,1rtkit-daemon0-21swapper/022:15:520
19089880,1rtkit-daemon0-21swapper/019:58:080
2645526271,3sleep12645578-21ssh00:35:001
2635803270,1sleep20-21swapper/200:26:042
2621049270,4sleep22621057-21ssh00:19:142
2562658271,3sleep22562859-21unixbench-2d23:40:232
2557208270,4sleep02557361-21df23:40:130
2549044270,5sleep02549101-21ls23:35:120
2526788271,3sleep20-21swapper/223:20:202
2508310270,4sleep12508502-21sed23:10:011
2500609271,3sleep00-21swapper/023:05:150
2491866271,3sleep22491874-21ssh22:59:242
2483735271,1sleep1171rcu_preempt22:54:301
2482245270,1sleep20-21swapper/222:50:582
2465319271,4sleep12465328-21ssh22:40:231
2449629270,5sleep22449637-21ssh22:31:032
2433340270,1sleep10-21swapper/122:20:561
2427711270,5sleep22427842-21ps22:20:172
2425242270,5sleep12313527-21diskmemload22:15:551
2419772271,4sleep22420763-21ssh22:15:162
2417404271,4sleep12417411-21sh22:11:211
2416642270,4sleep22416819-21grep22:10:272
2395545270,4sleep12396471-21cut22:00:161
2395006270,5sleep22395014-21ssh22:00:052
2378865271,3sleep10-21swapper/121:50:141
2378438270,1sleep20-21swapper/221:49:552
2330071270,4sleep22330112-21apt-get21:20:142
2329707270,4sleep12329777-21munin-run21:20:011
2321296270,5sleep12321302-21ssh21:14:201
2294659270,4sleep10-21swapper/120:55:131
2292267270,3sleep12293370-21memory20:50:191
2287288271,3sleep12287413-21python320:45:231
2277307270,4sleep02278508-21cut20:40:170
2244533271,3sleep02244622-21irqcore20:15:170
2244533271,3sleep02244622-21irqcore20:15:170
2242849270,4sleep02242993-21ntpq20:10:240
2231345271,3sleep12231361-21munin-run20:05:001
2190118270,4sleep22192245-21kthreadcore19:30:192
2161760270,5sleep12163597-21kthreadcore19:10:191
2154033270,4sleep12154186-21df_inode19:05:141
19089870,4rtkit-daemon674-21lldpd22:26:322
19089870,4rtkit-daemon671-21ntpd20:45:320
19089870,4rtkit-daemon2166508-21munin-node21:15:220
19089870,3rtkit-daemon671-21ntpd00:25:110
19089870,3rtkit-daemon2166508-21munin-node20:30:420
19089870,3rtkit-daemon2166508-21munin-node19:25:470
19089870,3rtkit-daemon1928-21sendmail-mta22:30:070
19089870,3rtkit-daemon1223-21cron20:59:000
2635608260,1sleep10-21swapper/100:25:391
2626999260,1sleep10-21swapper/100:20:221
2603525261,3sleep22603533-21ssh00:06:082
2603208260,1sleep10-21swapper/100:05:271
2578476261,2sleep20-21swapper/223:50:212
2548983260,1sleep10-21swapper/123:35:091
2542483262,2sleep22269319-21kworker/2:1-events23:30:182
2522628260,1sleep10-21swapper/123:15:451
2515510260,1sleep10-21swapper/123:12:571
2465925260,1sleep20-21swapper/222:40:412
2458215260,3sleep22458223-21ssh22:37:072
2435088260,1sleep10-21swapper/122:25:011
2409619260,1sleep20-21swapper/222:07:062
2409029260,3sleep12409039-21ssh22:05:411
2393437260,1sleep20-21swapper/221:57:042
2392844260,1sleep10-21swapper/121:55:431
2385370260,3sleep22385378-21ssh21:52:162
2353349260,3sleep22353357-21ssh21:33:282
2276132260,3sleep22276309-21cat20:40:152
2263019260,3sleep12263040-21munin-run20:30:001
2263019260,3sleep12263040-21munin-run20:30:001
2245973260,3sleep20-21swapper/220:15:192
2245973260,3sleep20-21swapper/220:15:182
2234217260,3sleep22236356-21cut20:05:172
2219205261,3sleep12219401-21hddtemp_smartct19:55:141
2211432260,3sleep02211572-21awk19:45:200
2207941262,2sleep20-21swapper/219:45:182
2199710260,1sleep20-21swapper/219:37:412
2180941260,1sleep10-21swapper/119:23:211
19089860,3rtkit-daemon81364-21packagekitd19:15:060
2652913250,3sleep10-21swapper/100:38:281
2613606250,1sleep10-21swapper/100:15:131
2582436250,2sleep12583614-21kthreadcore23:55:161
2572396250,3sleep22524287-21kworker/u16:1+rpciod23:48:492
2559801250,2sleep10-21swapper/123:40:181
2539316250,3sleep10-21swapper/123:27:001
2499600250,3sleep20-21swapper/223:03:332
2450322250,3sleep12450328-21ssh22:32:381
2377032250,3sleep10-21swapper/121:46:381
2369352250,2sleep12369360-21ssh21:42:371
2361275250,3sleep20-21swapper/221:37:372
2345067250,3sleep10-21swapper/121:28:021
2336695251,2sleep10-21swapper/121:22:291
2320007250,3sleep20-21swapper/221:11:202
2288360250,2sleep20-21swapper/220:50:132
2278509251,2sleep10-21swapper/120:40:181
2275445250,1sleep20-21swapper/220:38:232
2257398250,2sleep20-21swapper/220:25:172
2257398250,2sleep20-21swapper/220:25:172
2256595250,1sleep20-21swapper/220:23:082
2243539250,1sleep10-21swapper/120:10:411
2238071251,2sleep20-21swapper/220:10:182
2212167250,1sleep10-21swapper/119:46:091
2200209250,3sleep22200359-21df_inode19:40:122
2200173250,2sleep10-21swapper/119:40:121
2180751250,1sleep20-21swapper/219:20:542
2170171251,2sleep10-21swapper/119:15:151
2166093250,1sleep20-21swapper/219:11:522
2583616240,2sleep20-21swapper/223:55:182
2490893240,2sleep12490901-21ssh22:57:151
2296125240,2sleep20-21swapper/220:55:182
2263673240,2sleep10-21swapper/120:30:161
2230665240,2sleep20-21swapper/220:00:242
2224804240,3sleep20-21swapper/219:56:562
2218622240,3sleep10-21swapper/119:53:261
2186698240,2sleep10-21swapper/119:25:251
2168872240,2sleep22169026-21/usr/sbin/munin19:15:132
2157054240,1sleep20-21swapper/219:05:192
2495606230,1sleep10-21swapper/123:00:191
2307518230,1sleep20-21swapper/221:05:112
2286451230,1sleep20-21swapper/220:45:202
2231636230,1sleep10-21swapper/120:05:121
2212918230,1sleep20-21swapper/219:50:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional