You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-03 - 17:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackfslot7s.osadl.org (updated Sun May 03, 2026 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
138623999318612,3169cyclictest1391378-21kworker/0:2+events20:35:140
138624999304114,3022cyclictest1394392-21kworker/3:2+events19:35:123
138623999285214,2833cyclictest1435193-21kworker/0:2+events21:45:120
13862399928150,2811cyclictest1391378-21kworker/0:2+events19:30:160
138623999280413,2787cyclictest1460902-21kworker/0:2+events23:02:110
138623999279612,2780cyclictest1479656-21kworker/0:2+events00:17:050
138624999276612,2750cyclictest1447820-21kworker/3:0+events22:19:473
138623999276412,2748cyclictest1456119-21kworker/0:1+events22:48:490
138624999271913,2702cyclictest1396025-21kworker/3:0+events19:41:593
138624599271210,2700cyclictest1435317-21kworker/2:2+events21:40:122
138623999271013,2692cyclictest1425517-21kworker/0:0+events21:15:130
138623999270912,2693cyclictest1391378-21kworker/0:2+events19:50:490
138623999268413,2668cyclictest1402413-21kworker/0:0+events20:00:180
138624599267713,2661cyclictest1455999-21kworker/2:0+events22:43:042
138624999267613,2659cyclictest1455217-21kworker/3:2+events22:41:093
138623999267613,2661cyclictest1479656-21kworker/0:2+events00:10:140
138624999267313,2658cyclictest1447820-21kworker/3:0+events22:45:023
138624599266412,2649cyclictest1470348-21kworker/2:2+events23:39:052
138624599266212,2646cyclictest1444559-21kworker/2:1+events22:19:262
138624999266112,2645cyclictest1435256-21kworker/3:1+events21:47:093
138624999266110,2647cyclictest1435256-21kworker/3:1+events21:58:393
138623999265412,2639cyclictest1430417-21kworker/0:1+events21:36:450
138624999264913,2632cyclictest1449503-21kworker/3:1+events22:25:293
138624599264913,2632cyclictest1418751-21kworker/2:0+events20:47:222
138624599264313,2626cyclictest1438589-21kworker/2:3+events21:58:142
138624599261910,2604cyclictest1388603-21kworker/2:0+events19:20:212
138624999261713,2601cyclictest1402594-21kworker/3:2+events20:06:443
138624599259612,2581cyclictest1450867-21kworker/2:1+events22:28:102
138623999257712,2561cyclictest1471922-21kworker/0:1+events23:38:190
138624999257512,2559cyclictest1400932-21kworker/3:0+events19:57:183
138623999257412,2558cyclictest1481317-21kworker/0:1+events00:06:230
138623999257312,2557cyclictest1391378-21kworker/0:2+events20:26:230
138623999257012,2556cyclictest1408986-21kworker/0:0+events20:21:060
138623999257012,2554cyclictest1420451-21kworker/0:2+events21:06:100
138623999257011,2555cyclictest1418843-21kworker/0:0+events20:48:540
138623999256613,2549cyclictest1444684-21kworker/0:2+events22:09:400
13862459925649,2551cyclictest1430230-21kworker/2:0+events21:36:222
138624599256212,2546cyclictest1452732-21kworker/2:2+events22:37:202
138624599256112,2546cyclictest1426907-21kworker/2:0+events21:14:262
138623999256112,2546cyclictest1478014-21kworker/0:0+events23:50:530
138623999256012,2546cyclictest1418843-21kworker/0:0+events21:00:020
138623999255912,2544cyclictest1433556-21kworker/0:0+events21:42:300
138623999255911,2544cyclictest1448093-21kworker/0:3+events22:34:380
138623999255312,2537cyclictest1479656-21kworker/0:2+events00:21:100
138623999255312,2537cyclictest1447941-21kworker/0:0+events22:23:570
138623999254412,2528cyclictest1450929-21kworker/0:1+events22:29:190
138624999254013,2523cyclictest1472077-21kworker/3:3+events23:38:423
13862399925399,2526cyclictest1425517-21kworker/0:0+events21:24:010
138624999253511,2520cyclictest1404240-21kworker/3:0+events20:15:273
138624599253312,2517cyclictest1442965-21kworker/2:0+events22:03:072
138624599252112,2505cyclictest1402735-21kworker/2:3+events20:41:192
138624999251911,2504cyclictest1435256-21kworker/3:1+events21:44:483
138623999251212,2496cyclictest1473559-21kworker/0:2+events23:43:190
138624599251012,2494cyclictest1435317-21kworker/2:2+events21:52:552
138624999250911,2495cyclictest1447820-21kworker/3:0+events22:23:333
138624599250511,2488cyclictest1422347-21kworker/2:2+events21:03:522
138624199250411,2490cyclictest1486418-21kworker/1:0+events00:26:291
138624599250011,2485cyclictest1478932-21kworker/2:0+events00:00:412
138624999249910,2484cyclictest1387372-21kworker/3:0+events19:24:563
138623999249412,2479cyclictest1448093-21kworker/0:3+events22:40:460
138623999249412,2479cyclictest1405728-21kworker/0:0+events20:13:370
138623999249312,2477cyclictest1459297-21kworker/0:0+events23:05:130
138623999249211,2477cyclictest1479656-21kworker/0:2+events23:57:180
138624599248812,2474cyclictest1397877-21kworker/2:2+events19:45:052
138623999248712,2472cyclictest1448093-21kworker/0:3+events22:36:580
138623999248711,2472cyclictest1459297-21kworker/0:0+events23:26:520
138623999248512,2470cyclictest1459297-21kworker/0:0+events22:56:530
138623999248412,2469cyclictest1444684-21kworker/0:2+events22:18:150
138623999248412,2468cyclictest1443021-21kworker/0:0+events22:02:000
138623999248410,2470cyclictest1488108-21kworker/0:1+events00:28:430
138624999248210,2468cyclictest1446192-21kworker/3:1+events22:14:083
138623999248112,2466cyclictest1446340-21kworker/0:1+events22:14:300
138623999248111,2467cyclictest1418843-21kworker/0:0+events20:51:360
138623999248110,2468cyclictest1391378-21kworker/0:2+events19:26:490
138623999247812,2463cyclictest1476428-21kworker/0:1+events23:46:190
138623999247712,2462cyclictest1391378-21kworker/0:2+events20:19:580
138623999247712,2461cyclictest1456119-21kworker/0:1+events22:54:100
138623999247512,2460cyclictest1391378-21kworker/0:2+events20:40:320
138623999247412,2459cyclictest1433556-21kworker/0:0+events21:34:270
138623999247412,2459cyclictest1399240-21kworker/0:1+events19:48:560
138624599246812,2452cyclictest1476783-21kworker/2:1+events23:53:562
138624999246311,2448cyclictest1464037-21kworker/3:1+events23:10:333
138624599246311,2449cyclictest1396201-21kworker/2:1+events19:40:512
138624999246110,2447cyclictest1462346-21kworker/3:2+events23:07:063
138624599246111,2447cyclictest1430230-21kworker/2:0+events21:30:142
138623999245912,2444cyclictest1436850-21kworker/0:1+events21:51:240
138623999245412,2440cyclictest1423914-21kworker/0:1+events21:10:180
13862459924477,2437cyclictest1375011-21kworker/2:1+events19:30:402
138624599244710,2434cyclictest1438589-21kworker/2:3+events22:06:112
138624599244612,2430cyclictest1420421-21kworker/2:1+events20:54:202
138624599244412,2429cyclictest1405547-21kworker/2:0+events20:09:232
138624599244310,2429cyclictest1462592-21kworker/2:1+events23:06:432
138624599244012,2424cyclictest1475157-21kworker/2:0+events23:45:132
138624599244012,2424cyclictest1402735-21kworker/2:3+events20:35:362
138624999243910,2425cyclictest1459087-21kworker/3:0+events22:55:193
13862499924387,2427cyclictest1420496-21kworker/3:1+events21:09:313
13862459924385,2429cyclictest1394572-21kworker/2:2+events19:35:592
138624599243811,2424cyclictest1402735-21kworker/2:3+events20:30:382
138624599243811,2423cyclictest1452732-21kworker/2:2+events22:32:192
13862499924366,2427cyclictest1413782-21kworker/3:1+events20:40:093
138624599243512,2420cyclictest1422347-21kworker/2:2+events21:09:542
138624599243412,2418cyclictest1409067-21kworker/2:2+events20:29:512
138624999243310,2419cyclictest1488165-21kworker/3:1+events00:35:123
138623999243312,2418cyclictest1462648-21kworker/0:1+events23:16:400
13862499924284,2420cyclictest1425337-21kworker/3:0+events21:11:503
138624999242810,2415cyclictest1471713-21kworker/3:1+events23:40:143
138623999242811,2414cyclictest1481317-21kworker/0:1+events00:02:360
138624599242712,2411cyclictest1465428-21kworker/2:0+events23:17:462
13862499924267,2416cyclictest1411909-21kworker/3:2+events20:29:283
138624599242612,2410cyclictest1481224-21kworker/2:2+events00:08:422
138624599242612,2410cyclictest1450867-21kworker/2:1+events22:51:522
138624999242512,2409cyclictest1427241-21kworker/3:3+events21:18:403
138624599242512,2410cyclictest1470348-21kworker/2:2+events23:34:062
138624999242412,2409cyclictest1394392-21kworker/3:2+events19:32:113
138624999242412,2408cyclictest1404240-21kworker/3:0+events20:37:513
138624999242312,2407cyclictest1404240-21kworker/3:0+events20:32:113
13862499924219,2408cyclictest1425337-21kworker/3:0+events21:31:443
138624999242112,2405cyclictest1404240-21kworker/3:0+events20:48:303
138624999242112,2405cyclictest1402594-21kworker/3:2+events20:02:333
138624599242112,2405cyclictest1375011-21kworker/2:1+events19:27:112
138624999242012,2406cyclictest1465641-21kworker/3:2+events23:20:053
138624999242012,2406cyclictest1420496-21kworker/3:1+events20:55:043
138624599242012,2404cyclictest1449267-21kworker/2:0+events22:22:242
138624999241511,2401cyclictest1425337-21kworker/3:0+events21:20:363
138624999241412,2399cyclictest1488165-21kworker/3:1+events00:27:363
138624599241312,2397cyclictest1459238-21kworker/2:0+events23:03:412
138623999241212,2397cyclictest1459297-21kworker/0:0+events23:23:060
138624999241110,2397cyclictest1404240-21kworker/3:0+events20:50:283
138624599241112,2395cyclictest1491011-21kworker/2:0+events00:36:432
138624599241111,2396cyclictest1407150-21kworker/2:1+events20:15:502
138623999240911,2395cyclictest1385930-21kworker/0:0+events19:21:310
138624999240811,2393cyclictest1435256-21kworker/3:1+events22:05:253
138623999240512,2390cyclictest1387638-21kworker/0:2+events19:17:210
13862499924026,2392cyclictest1425337-21kworker/3:0+events21:27:073
138624199240010,2386cyclictest1441698-21kworker/1:1+events22:03:541
13862459923998,2388cyclictest1435317-21kworker/2:2+events21:45:352
138623999239711,2383cyclictest1391378-21kworker/0:2+events19:43:090
138624999239611,2382cyclictest1441400-21kworker/3:0+events22:01:163
138624599239212,2376cyclictest1450867-21kworker/2:1+events22:46:312
138624599238211,2368cyclictest1470348-21kworker/2:2+events23:44:272
138624999238012,2365cyclictest1465641-21kworker/3:2+events23:25:223
138624999237612,2360cyclictest1422958-21kworker/3:2+events21:04:153
138624999237611,2362cyclictest1408253-21kworker/3:2+events20:20:213
138624599237612,2361cyclictest1488434-21kworker/2:1+events00:29:282
138624199237611,2361cyclictest1432012-21kworker/1:0+events22:51:051
138624999237512,2360cyclictest1451094-21kworker/3:2+events22:31:563
138624599237512,2360cyclictest1482917-21kworker/2:1+events00:16:202
138624999237312,2357cyclictest1465641-21kworker/3:2+events23:18:103
138623999237312,2358cyclictest1479656-21kworker/0:2+events00:30:160
138624599237212,2356cyclictest1470348-21kworker/2:2+events23:29:302
138624599237012,2355cyclictest1444916-21kworker/2:2+events22:11:302
138624599237012,2354cyclictest1481224-21kworker/2:2+events00:10:582
138624999236912,2354cyclictest1486509-21kworker/3:2+events00:22:443
138624599236812,2352cyclictest1399597-21kworker/2:1+events19:59:312
138623999236812,2352cyclictest1479656-21kworker/0:2+events00:35:340
138624999236611,2352cyclictest1490981-21kworker/3:2+events00:34:253
138624599236411,2351cyclictest1418751-21kworker/2:0+events20:55:502
138624999236212,2348cyclictest1459087-21kworker/3:0+events23:31:043
138624599236212,2348cyclictest1401350-21kworker/2:2+events20:01:272
138624199236212,2346cyclictest1470136-21kworker/1:1+events00:18:121
138624599236112,2346cyclictest1465428-21kworker/2:0+events23:23:272
138624199236012,2345cyclictest1385779-21kworker/1:0+events20:48:081
138624999235712,2341cyclictest1483070-21kworker/3:1+events00:13:163
138624999235612,2341cyclictest1476572-21kworker/3:2+events23:59:133
138624999235512,2340cyclictest1400932-21kworker/3:0+events19:54:593
138624999235511,2341cyclictest1457481-21kworker/3:1+events23:01:263
138624999235312,2337cyclictest1481467-21kworker/3:0+events00:19:193
138624999235112,2335cyclictest1387372-21kworker/3:0+events19:27:583
138624599235111,2336cyclictest1488434-21kworker/2:1+events00:34:492
138624999234911,2334cyclictest1476572-21kworker/3:2+events23:48:323
138624599234812,2333cyclictest1476783-21kworker/2:1+events23:58:492
138623999234412,2328cyclictest1402413-21kworker/0:0+events19:58:260
138624999234212,2327cyclictest1457481-21kworker/3:1+events22:51:293
138624999234212,2326cyclictest1481467-21kworker/3:0+events00:04:313
138624599234212,2326cyclictest1430230-21kworker/2:0+events21:25:562
138624599234113,2324cyclictest1462592-21kworker/2:1+events23:11:382
138624599234112,2326cyclictest1424087-21kworker/2:3+events21:20:582
138624599234012,2325cyclictest1459238-21kworker/2:0+events22:58:472
138624999233912,2324cyclictest1481467-21kworker/3:0+events00:09:053
138624999233811,2323cyclictest1404240-21kworker/3:0+events20:12:053
138624999233810,2325cyclictest1384517-21kworker/3:2+events19:16:343
138624999233711,2323cyclictest1396025-21kworker/3:0+events19:45:523
138624599233711,2323cyclictest1424087-21kworker/2:3+events21:17:312
138624599233612,2321cyclictest1409067-21kworker/2:2+events20:22:392
138624599232912,2314cyclictest1399597-21kworker/2:1+events19:52:422
138624599232710,2314cyclictest1402735-21kworker/2:3+events20:14:432
138624199232612,2311cyclictest1405699-21kworker/1:1+events20:09:461
138624999232512,2310cyclictest1435256-21kworker/3:1+events21:53:193
138623999232411,2310cyclictest1395935-21kworker/0:1+events19:38:130
138623999232212,2307cyclictest1391378-21kworker/0:2+events20:05:110
138623999232111,2307cyclictest1391378-21kworker/0:2+events20:33:210
138624599232012,2306cyclictest1486450-21kworker/2:2+events00:20:042
138623999232011,2306cyclictest1428695-21kworker/0:2+events21:29:490
138623999231912,2304cyclictest1420451-21kworker/0:2+events20:59:160
138624999231411,2299cyclictest1433526-21kworker/3:2+events21:36:003
138623999231310,2300cyclictest1435193-21kworker/0:2+events21:57:060
138623999226211,2247cyclictest1385930-21kworker/0:0+events19:14:140
138623999225810,2245cyclictest1470197-21kworker/0:2+events23:31:250
138623999223710,2225cyclictest1459297-21kworker/0:0+events23:10:080
138624999218410,2171cyclictest1447820-21kworker/3:0+events22:35:483
13862419921737,2162cyclictest1470136-21kworker/1:1+events23:59:351
138624599214912,2134cyclictest1388603-21kworker/2:0+events19:16:112
138624999214211,2127cyclictest1471713-21kworker/3:1+events23:54:183
138624599214111,2127cyclictest1379491-21kworker/2:2+events19:13:042
138624199214111,2127cyclictest1476870-21kworker/1:2+events00:05:171
138624199213510,2122cyclictest1385779-21kworker/1:0+events19:19:351
138624999213011,2116cyclictest1387372-21kworker/3:0+events19:14:383
138624199212010,2106cyclictest1464218-21kworker/1:2+events23:26:071
138624199211010,2096cyclictest1425666-21kworker/1:2+events21:30:591
138624199210611,2092cyclictest1432012-21kworker/1:0+events23:14:451
138624199210511,2091cyclictest1418874-21kworker/1:1+events21:12:351
138624199209510,2082cyclictest1387763-21kworker/1:2+events19:37:081
13857892190161,24sleep00-21swapper/019:06:390
13860392159116,25sleep20-21swapper/219:09:492
13857142150110,36sleep30-21swapper/319:05:423
13858402144108,31sleep10-21swapper/119:07:151
1386241997269,2cyclictest1432012-21kworker/1:0+pm21:59:591
1386241993125,4cyclictest710-21mta-sts-daemon22:20:001
138624199308,9cyclictest1414886-21nvmesmart_nvme020:35:191
138624199305,14cyclictest1391969-21memory19:25:161
1386241993014,12cyclictest1417221-21sleep20:45:001
138624199287,8cyclictest1421541-21memory20:55:151
138624199277,8cyclictest1408212-21ntpq20:15:161
138624199267,6cyclictest0-21swapper/120:30:021
1386241992620,1cyclictest1429582-21ntpq21:20:161
1386241992615,7cyclictest1435403-21awk21:40:001
1386241992614,8cyclictest1464143-21munin-run23:10:001
1386241992614,8cyclictest1444898-21awk22:10:001
138624199258,9cyclictest1490269-21smartctl00:30:151
138624199257,6cyclictest1392891-21apt-get19:30:031
138624199257,11cyclictest1492124-21ntpq00:35:181
138624199255,8cyclictest1420617-21latency_hist20:55:011
138624199255,13cyclictest1477462-21memory23:50:171
138624199254,8cyclictest1450408-21ntpq22:25:161
1386241992514,8cyclictest1425649-21basename21:10:001
1386241992514,8cyclictest1407446-21idleruntime-cro20:15:001
1386241992510,8cyclictest1410877-21apt-get20:25:091
138624199248,9cyclictest1428060-21ntpq21:15:191
138624199247,11cyclictest1439324-21ntpq21:50:181
138624199245,6cyclictest1400378-21ntpq19:50:181
138624199244,6cyclictest0-21swapper/122:10:171
1386241992418,4cyclictest593-21dbus-daemon23:39:591
1386241992414,6cyclictest740-21snmpd22:45:001
138624199237,9cyclictest1468147-21ntpq23:20:181
138624199236,6cyclictest1432292-21cat21:30:011
138624199236,5cyclictest1398942-21python319:45:211
138624199235,5cyclictest1452297-21ntpq22:30:181
1386241992317,2cyclictest1484739-21apt-get00:15:011
1386241992315,5cyclictest1467318-21basename23:20:001
1386241992313,7cyclictest1391324-21awk19:25:011
138624199227,8cyclictest1471266-21ntpq23:30:161
138624199227,4cyclictest1453847-21ntpq22:35:171
138624199226,9cyclictest1448835-21ntpq22:20:171
138624199226,8cyclictest1461474-21hddtemp_smartct23:00:131
138624199226,6cyclictest1457004-21ntpq22:45:171
138624199226,6cyclictest1410022-21python320:20:191
138624199225,6cyclictest1476037-21ntpq23:45:171
138624199225,11cyclictest1460202-21ntpq22:55:181
138624199223,6cyclictest1397191-21smartctl19:40:181
1386241992216,4cyclictest1474054-21ls23:40:111
138624199216,9cyclictest1401913-21ntpq19:55:181
138624199206,4cyclictest1480993-21python300:00:191
138624199205,5cyclictest1403486-21ntpq20:00:171
138624199205,4cyclictest1422943-21df21:00:141
138624199204,6cyclictest1437776-21ntpq21:45:171
138624199188,7cyclictest1486877-21apt00:20:131
138624199185,7cyclictest1435880-21diskstats21:40:131
138624199174,8cyclictest1386635-21hddtemp_smartct19:10:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional