You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-15 - 12:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot8s.osadl.org (updated Sun Mar 15, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21027339911381,1132cyclictest2428474-21sleep23:10:013
2102733998573,850cyclictest2552010-21sleep00:34:593
2102733996641,658cyclictest2305693-21sleep21:45:003
2102733993080,304cyclictest2108900-21turbostat19:19:553
25398982500,1sleep30-21swapper/300:25:173
72699270,4ntpd2528204-21taskset00:15:232
72699250,9ntpd2203647-21ntpq20:25:211
72699240,7ntpd2269229-21apt-get21:20:030
72699240,7ntpd2241016-21ntpq20:55:201
72699240,7ntpd2128709-21ntpq19:25:202
72699240,6ntpd2269816-21ssh21:20:102
72699230,8ntpd2254680-21diskmemload21:45:442
72699230,7ntpd2419389-21ntpq23:00:191
72699230,7ntpd2419389-21ntpq23:00:191
72699230,6ntpd2228621-21ntpq20:45:211
72699230,6ntpd2161131-21apt-get19:55:110
72699230,6ntpd2102921-21apt-get19:10:121
72699230,2ntpd0-21swapper/120:00:211
72699220,9ntpd2406425-21taskset22:53:300
72699220,8ntpd2412319-21ntpq22:55:190
72699220,8ntpd2254680-21diskmemload22:25:221
72699220,7ntpd2537465-21latency_hist00:25:002
72699220,6ntpd2557443-21ntp_states00:35:210
72699220,6ntpd2448427-21ntpq23:20:210
72699220,6ntpd2101719-21ntp_states19:05:200
72699220,5ntpd2535348-21ntpq00:20:200
72699220,5ntpd2433883-21ntpq23:10:230
72699220,5ntpd2262305-21apt-get21:15:120
72699220,5ntpd2255001-21apt-get21:10:122
72699220,5ntpd2134915-21ntpq19:30:201
72699220,2ntpd0-21swapper/220:20:202
72699220,2ntpd0-21swapper/021:55:200
72699210,7ntpd2514675-21ssh00:07:030
72699210,7ntpd2234715-21grep20:50:190
72699210,7ntpd2141159-21ntpq19:35:200
72699210,6ntpd2528197-21ntpq00:15:231
72699210,6ntpd2134966-21ntpq19:30:210
72699210,5ntpd2520937-21ntp_states00:10:211
72699210,5ntpd2520874-21ntpq00:10:202
72699210,5ntpd2494090-21apt-get23:55:111
72699210,5ntpd2479634-21apt-get23:45:121
72699210,5ntpd2378165-21apt-get22:35:120
72699210,5ntpd2349312-21apt-get22:15:141
72699210,5ntpd2347240-21ntpq22:10:200
72699210,5ntpd2334852-21apt-get22:05:112
72699210,5ntpd2224890-21irqstats20:45:152
72699210,5ntpd2172941-21users20:00:252
72699210,5ntpd2147437-21ntpq19:40:192
72699210,5ntpd2117464-21apt-get19:20:121
72699210,5ntpd2108003-21ntpq19:10:212
72699210,4ntpd2515786-21apt-get00:10:120
72699210,4ntpd2385428-21apt-get22:40:090
72699210,4ntpd2284256-21apt-get21:30:112
72699210,2ntpd0-21swapper/223:45:212
72699210,2ntpd0-21swapper/121:00:211
72699200,7ntpd2464105-21ssh23:32:272
72699200,5ntpd2558772-21ssh00:38:262
72699200,5ntpd2545186-21df_inode00:30:140
72699200,5ntpd2484808-21ntp_states23:45:220
72699200,5ntpd2470239-21ntpq23:35:210
72699200,5ntpd2399897-21apt-get22:50:131
72699200,5ntpd2383268-21ntpq22:35:201
72699200,5ntpd2361669-21ntpq22:20:191
72699200,5ntpd2335013-21df_inode22:05:130
72699200,5ntpd2325507-21ntpq21:55:212
72699200,5ntpd2313200-21apt-get21:50:112
72699200,5ntpd2291447-21ssh21:35:121
72699200,5ntpd2260180-21ntpq21:10:211
72699200,5ntpd2254328-21sshd21:05:370
72699200,5ntpd2216090-21ntpq20:35:201
72699200,5ntpd2191344-21awk20:15:251
72699200,5ntpd2185846-21munin-run20:15:001
72699200,5ntpd2184965-21ntpq20:10:210
72699200,5ntpd2136127-21/usr/sbin/munin19:35:141
72699200,5ntpd2128811-21ntp_states19:25:220
72699200,5ntpd2109085-21munin-node19:15:011
72699200,4ntpd2552305-21apt-get00:35:111
72699200,4ntpd2523019-21apt-get00:15:120
72699200,4ntpd2465118-21ssh23:35:132
72699200,4ntpd2413685-21ssh22:58:461
72699200,4ntpd2267411-21ssh21:15:201
72699200,4ntpd2211032-21apt-get20:35:110
72699200,3ntpd2371021-21ssh22:30:122
72699200,3ntpd2368981-21ssh22:25:222
72699190,6ntpd2437200-21irqrtprio23:15:192
72699190,6ntpd2415432-21sed23:00:160
72699190,6ntpd2415432-21sed23:00:150
72699190,6ntpd2219360-21cut20:40:162
72699190,6ntpd2219360-21cut20:40:162
72699190,6ntpd2192713-21/usr/sbin/munin20:20:130
72699190,5ntpd804-21runrttasks22:26:240
72699190,5ntpd2374437-21kthreadcore22:30:170
72699190,5ntpd2349469-21df22:15:140
72699190,5ntpd2236104-21df20:55:130
72699190,5ntpd2231559-21grep20:50:172
72699190,5ntpd2209840-21ntpq20:30:220
72699190,5ntpd2181197-21irqstats20:10:152
72699190,4ntpd2508855-21perl00:05:142
72699190,4ntpd2478354-21ssh23:41:261
72699190,4ntpd2421817-21fw_packets23:05:140
72699190,4ntpd2412269-21ld22:55:192
72699190,4ntpd2356548-21apt-get22:20:110
72699190,4ntpd2327616-21apt-get22:00:111
72699190,4ntpd2289445-21ssh21:30:211
72699190,4ntpd2241911-21apt-get21:00:012
72699190,3ntpd2435991-21ssh23:15:151
72699190,3ntpd2389906-21kworker/0:1+mm_percpu_wq23:40:210
72699190,3ntpd2282203-21ssh21:25:221
2102733991916,1cyclictest0-21swapper/322:30:163
72699180,7ntpd2464164-21ssh23:32:391
72699180,5ntpd2450797-21wc23:25:121
72699180,5ntpd2345051-21cut22:10:162
72699180,5ntpd2311243-21/usr/sbin/munin21:45:220
72699180,5ntpd2267514-21ntpq21:15:222
72699180,5ntpd2255828-21irqcore21:10:150
72699180,5ntpd2248987-21grep21:05:142
72699180,5ntpd2199158-21cut20:25:150
72699180,5ntpd2103273-21/usr/sbin/munin19:10:150
72699180,5ntpd0-21swapper/221:37:442
72699180,5ntpd0-21swapper/200:00:192
72699180,5ntpd0-21swapper/119:40:211
72699180,4ntpd2543038-21ps00:25:232
72699180,4ntpd2492817-21ssh23:51:252
72699180,4ntpd2488238-21kthreadcore23:50:170
72699180,4ntpd2428992-21grep23:10:151
72699180,4ntpd2420750-21ssh23:03:132
72699180,4ntpd2420750-21ssh23:03:132
72699180,4ntpd2384468-21ssh22:37:382
72699180,4ntpd2319927-21ssh21:54:280
72699180,4ntpd2319896-21ssh21:54:211
72699180,4ntpd2305603-21ssh21:44:470
72699180,4ntpd2160856-21latency_hist19:55:010
72699180,4ntpd2148813-21df19:45:131
72699180,3ntpd2282569-21unixbench-2d21:25:250
72699180,2ntpd0-21swapper/120:30:211
72699180,2ntpd0-21swapper/100:05:121
72699180,2ntpd0-21swapper/023:15:250
210273399187,7cyclictest24048922sleep322:50:203
72699170,6ntpd808-21snmpd21:39:310
72699170,6ntpd2478892-21ssh23:43:302
72699170,5ntpd2362483-21ssh22:21:152
72699170,5ntpd2325560-21ssh21:55:221
72699170,4ntpd2550934-21ssh00:31:122
72699170,4ntpd2544341-21ssh00:28:421
72699170,4ntpd2532351-21cut00:20:171
72699170,4ntpd2499891-21ssh23:55:482
72699170,4ntpd2499218-21ntpq23:55:200
72699170,4ntpd2462994-21ntpq23:30:210
72699170,4ntpd2406587-21ssh22:54:062
72699170,4ntpd2398692-21sshd22:46:390
72699170,4ntpd2392139-21ssh22:44:182
72699170,4ntpd2391818-21ssh22:43:031
72699170,4ntpd2376707-21ssh22:30:451
72699170,4ntpd2351899-21grep22:15:192
72699170,4ntpd2348334-21ssh22:12:331
72699170,4ntpd2334112-21ssh22:03:242
72699170,4ntpd2327668-21/usr/sbin/munin22:00:120
72699170,4ntpd2291050-21sshd21:34:370
72699170,4ntpd2251676-21ps21:05:171
72699170,4ntpd2200077-21grep20:25:162
72699170,4ntpd2109085-21munin-node19:15:030
72699170,3ntpd2471197-21ssh23:36:451
72699170,3ntpd2298774-21apt-get21:40:112
72699170,3ntpd2276545-21ssh21:24:231
72699170,3ntpd2154872-21apt-get19:50:111
72699170,2ntpd0-21swapper/100:30:131
72699170,1ntpd0-21swapper/219:55:202
2102733991714,1cyclictest0-21swapper/322:05:013
210273399171,13cyclictest2148390-21turbostat19:45:003
72699160,6ntpd2339652-21kthreadcore22:05:171
72699160,5ntpd0-21swapper/222:48:442
72699160,4ntpd2506791-21cat00:00:251
72699160,4ntpd2456418-21ssh23:25:342
72699160,4ntpd2448936-21ssh23:20:292
72699160,4ntpd2434503-21ssh23:10:432
72699160,4ntpd2311575-21unixbench_multi21:45:241
72699160,4ntpd2282765-21ssh21:25:392
72699160,4ntpd2254680-21diskmemload23:06:422
72699160,4ntpd2150041-21kthreadcore19:45:182
72699160,3ntpd2540268-21cut00:25:170
72699160,3ntpd2508136-21taskset00:04:350
72699160,3ntpd2491948-21ssh23:50:211
72699160,3ntpd2452971-21kthreadcore23:25:160
72699160,3ntpd2448397-21ssh23:20:211
72699160,3ntpd2398921-21ssh22:47:341
72699160,3ntpd2301936-21kthreadcore21:40:161
72699160,3ntpd2186385-21smartctl20:15:160
72699160,3ntpd2177976-21grep20:05:161
72699160,2ntpd2391139-21kworker/u16:0+rpciod23:05:251
72699160,2ntpd0-21swapper/219:15:232
72699160,2ntpd0-21swapper/120:40:211
72699160,2ntpd0-21swapper/120:40:201
210273399165,7cyclictest25370572sleep300:24:293
2102733991613,1cyclictest0-21swapper/322:36:143
210273399161,12cyclictest2341959-21taskset22:10:003
210273399161,10cyclictest2241937-21turbostat21:00:003
72699150,3ntpd2224923-21kthreadcore20:45:160
72699150,3ntpd2195309-21kthreadcore20:20:161
72699150,3ntpd2175830-21cat20:05:150
72699150,2ntpd2147375-21strings19:40:180
72699150,2ntpd0-21swapper/220:35:212
72699150,2ntpd0-21swapper/220:15:222
72699150,2ntpd0-21swapper/220:05:182
72699150,2ntpd0-21swapper/219:50:202
72699150,2ntpd0-21swapper/219:35:202
72699150,2ntpd0-21swapper/219:30:242
72699150,2ntpd0-21swapper/219:20:212
72699150,2ntpd0-21swapper/119:05:211
72699150,2ntpd0-21swapper/020:40:220
72699150,2ntpd0-21swapper/020:40:210
72699150,2ntpd0-21swapper/019:45:120
72699150,2ntpd0-21swapper/019:20:200
72699150,1ntpd0-21swapper/119:25:121
210273399155,8cyclictest2442895-21turbostat23:20:003
210273399155,7cyclictest24579102sleep323:30:123
210273399155,7cyclictest23919812sleep322:43:443
210273399154,8cyclictest2192326-21taskset20:20:013
210273399154,8cyclictest2154579-21taskset19:49:563
210273399153,9cyclictest2204526-21turbostat20:30:013
210273399151,9cyclictest23637752sleep322:25:123
2102733991512,1cyclictest0-21swapper/323:28:313
2102733991512,1cyclictest0-21swapper/320:47:043
2102733991512,1cyclictest0-21swapper/320:37:503
2102733991512,1cyclictest0-21swapper/320:15:003
210273399151,11cyclictest2492880-21taskset23:51:383
210273399151,11cyclictest2254919-21taskset21:10:013
210273399151,10cyclictest2494093-21chrt23:55:113
210273399151,10cyclictest2248880-21taskset21:05:133
72699140,2ntpd0-21swapper/220:30:222
72699140,2ntpd0-21swapper/021:00:160
72699140,1ntpd0-21swapper/219:05:202
72699140,1ntpd0-21swapper/020:00:190
210273399145,6cyclictest24139902sleep322:59:553
210273399143,8cyclictest2443487-21taskset23:20:143
210273399141,9cyclictest2269249-21turbostat21:20:003
210273399141,9cyclictest2159956-21taskset19:50:203
210273399141,8cyclictest25522812sleep300:35:113
2102733991413,0cyclictest0-21swapper/322:12:343
2102733991412,1cyclictest0-21swapper/321:57:403
2102733991412,1cyclictest0-21swapper/321:45:203
210273399141,11cyclictest2508497-21taskset00:05:013
2102733991411,1cyclictest0-21swapper/323:44:333
2102733991411,1cyclictest0-21swapper/323:36:553
2102733991411,1cyclictest0-21swapper/323:13:473
2102733991411,1cyclictest0-21swapper/321:50:123
2102733991411,1cyclictest0-21swapper/320:30:363
2102733991411,1cyclictest0-21swapper/319:34:253
210273399141,10cyclictest2522035-21chrt00:12:303
210273399141,10cyclictest2362439-21taskset22:21:063
210273399141,10cyclictest2289949-21chrt21:30:263
210273399141,10cyclictest2242476-21chrt21:00:113
210273399141,10cyclictest2161090-21taskset19:55:043
210273399140,11cyclictest2479556-21taskset23:45:023
210273399140,11cyclictest2291406-21taskset21:35:013
72699130,2ntpd0-21swapper/119:56:311
72699130,1ntpd0-21swapper/220:55:152
72699130,1ntpd0-21swapper/120:50:201
210273399135,6cyclictest23552092sleep322:16:153
210273399134,7cyclictest2282086-21taskset21:25:203
210273399134,7cyclictest2235329-21taskset20:50:393
210273399131,9cyclictest2123655-21taskset19:25:113
210273399131,9cyclictest2118283-21chrt19:20:143
210273399131,8cyclictest2523318-21chrt00:15:163
210273399131,10cyclictest2420959-21taskset23:03:583
210273399131,10cyclictest2420959-21taskset23:03:583
2102733991311,0cyclictest0-21swapper/320:18:293
2102733991310,1cyclictest0-21swapper/320:00:213
210273399130,5cyclictest0-21swapper/319:10:023
210273399130,10cyclictest2398622-21taskset22:46:233
210273399130,10cyclictest2275607-21chrt21:20:513
210273399130,10cyclictest2179274-21taskset20:06:143
210273399121,9cyclictest2142078-21taskset19:39:383
210273399121,8cyclictest2222936-21taskset20:40:533
210273399121,8cyclictest2222936-21taskset20:40:523
210273399120,10cyclictest2507107-21taskset00:00:373
2096385260,2sleep30-21swapper/319:05:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional