You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-17 - 23:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Wed Dec 17, 2025 12:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
154559321270,3sleep2574-21rs:main2
1299925212051,64sleep30-21swapper/307:08:283
13090842970,2sleep1321rcuc/107:15:151
129987228164,10sleep10-21swapper/107:07:501
130003027953,20sleep00-21swapper/007:09:500
129969427559,10sleep20-21swapper/207:05:352
1300167993226,3cyclictest1598632-21sleep11:50:003
1300167992219,1cyclictest0-21swapper/312:20:243
130016799213,11cyclictest1324105-21taskset07:30:123
1300167992118,1cyclictest0-21swapper/309:32:573
15254902201,8sleep01525751-21kthreadcore10:40:150
130016799202,11cyclictest1482205-21taskset09:58:163
1300167992017,1cyclictest0-21swapper/311:21:573
1300167992017,1cyclictest0-21swapper/308:55:173
1300167992017,1cyclictest0-21swapper/308:45:203
1300167992017,1cyclictest0-21swapper/308:35:153
1300167992017,1cyclictest0-21swapper/307:25:223
1300167992016,2cyclictest0-21swapper/310:05:013
130016799195,12cyclictest0-21swapper/309:07:103
130016799195,10cyclictest1524687-21turbostat10:40:003
130016799194,6cyclictest0-21swapper/309:40:113
130016799193,12cyclictest1493146-21taskset10:10:113
130016799193,10cyclictest1582397-21taskset11:33:443
1300167991916,1cyclictest0-21swapper/312:03:573
1300167991916,1cyclictest0-21swapper/309:53:573
1300167991916,1cyclictest0-21swapper/308:00:163
1300167991916,1cyclictest0-21swapper/307:51:573
1300167991916,1cyclictest0-21swapper/307:40:143
1300167991916,1cyclictest0-21swapper/307:35:573
1300167991915,2cyclictest0-21swapper/308:33:103
130016799191,15cyclictest1624708-21taskset12:10:273
130016799191,13cyclictest0-21swapper/311:15:123
130016799190,12cyclictest1540256-21chrt10:53:323
16101842180,7sleep01611634-21tr12:00:150
130016799185,9cyclictest1361036-21chrt08:05:113
130016799185,2cyclictest0-21swapper/310:08:523
130016799184,10cyclictest1302883-21chrt07:10:163
1300167991815,1cyclictest0-21swapper/311:55:113
1300167991815,1cyclictest0-21swapper/311:35:283
1300167991815,1cyclictest0-21swapper/311:25:153
1300167991815,1cyclictest0-21swapper/310:29:343
1300167991815,1cyclictest0-21swapper/309:26:573
1300167991815,1cyclictest0-21swapper/309:26:573
1300167991815,1cyclictest0-21swapper/308:51:573
1300167991814,2cyclictest0-21swapper/307:20:003
130016799181,14cyclictest0-21swapper/310:30:113
130016799181,14cyclictest0-21swapper/310:30:113
130016799181,13cyclictest1508864-21taskset10:24:503
130016799181,13cyclictest0-21swapper/312:30:183
130016799181,13cyclictest0-21swapper/311:10:103
130016799181,13cyclictest0-21swapper/307:55:203
130016799181,10cyclictest1369402-21taskset08:10:163
130016799180,14cyclictest1555993-21taskset11:09:513
130016799180,14cyclictest1439659-21taskset09:16:133
15549592170,1sleep20-21swapper/211:05:182
130016799176,1cyclictest0-21swapper/310:55:173
130016799174,9cyclictest1344725-21turbostat07:50:003
130016799172,9cyclictest1387146-21taskset08:29:313
1300167991714,1cyclictest0-21swapper/311:50:193
1300167991714,1cyclictest0-21swapper/308:20:183
1300167991714,1cyclictest0-21swapper/308:20:183
1300167991714,1cyclictest0-21swapper/307:20:193
1300167991713,2cyclictest0-21swapper/312:26:303
1300167991713,2cyclictest0-21swapper/310:45:093
1300167991713,1cyclictest0-21swapper/311:00:193
1300167991712,3cyclictest541ktimers/312:15:173
130016799170,13cyclictest0-21swapper/311:40:183
130016799161,9cyclictest1524946-21taskset10:40:053
130016799161,9cyclictest1466892-21taskset09:45:063
130016799161,13cyclictest0-21swapper/309:00:123
130016799161,10cyclictest1618794-21taskset12:05:183
1300167991610,2cyclictest0-21swapper/309:20:123
130016799160,13cyclictest0-21swapper/312:35:173
130016799160,13cyclictest0-21swapper/310:15:133
130016799160,13cyclictest0-21swapper/308:15:183
16404892151,4sleep11640647-21sed12:25:251
15463862151,8sleep01546627-21kthreadcore11:00:150
15149882153,8sleep21515022-21kthreadcore10:30:152
15149882153,8sleep21515022-21kthreadcore10:30:152
130016799151,9cyclictest1433747-21taskset09:10:203
130016799150,12cyclictest0-21swapper/308:40:253
15455822141,9sleep11545591-21cron11:00:001
15077822141,3sleep21507871-21ntpq10:20:172
14603432141,2sleep10-21swapper/109:35:201
14288502141,9sleep21429087-21/usr/sbin/munin09:05:272
13053412141,8sleep21305678-21irqstats07:15:012
1300167991411,2cyclictest0-21swapper/309:35:113
15931312131,8sleep11593578-21apt-get11:45:011
15202142131,7sleep01520482-21kthreadcore10:35:140
15092022131,8sleep21509420-21df_inode10:25:102
14510612130,1sleep00-21swapper/009:30:140
13705292131,8sleep21370576-21sort08:10:242
15833302121,7sleep21584562-21kthreadcore11:35:162
14925352120,1sleep10-21swapper/110:05:541
14302852125,3sleep21430504-21kthreadcore09:10:162
13558882121,5sleep11356011-21diskstats08:00:131
13402892121,3sleep21340788-21cut07:45:152
13288592121,7sleep21328946-21latency_hist07:35:012
13135332121,7sleep01314491-21kthreadcore07:20:160
16469952110,1sleep10-21swapper/112:35:071
16464652111,5sleep21646537-21munin-run12:35:002
16417872111,4sleep01641997-21hddtemp_smartct12:30:130
15827452110,5sleep10-21swapper/111:35:031
15654522111,5sleep11565500-21ntpq11:15:191
15568362111,6sleep11557911-21kthreadcore11:10:141
15026262111,4sleep21502674-21ntpq10:15:202
14918222110,3sleep20-21swapper/210:05:182
14773502112,4sleep01477369-21apt-get09:55:100
14035432111,5sleep21403752-21hddtemp_smartct08:45:152
13913312112,4sleep21391444-21ntpq08:30:202
13668102111,6sleep11367636-21kthreadcore08:10:151
13223392111,5sleep01322393-21ntpq07:25:210
13008742111,5sleep11301317-21kthreadcore07:10:151
16365002101,4sleep21636710-21hddtemp_smartct12:25:142
16359442101,4sleep01636118-21grep12:25:000
16256222101,4sleep01625878-21df12:15:100
16038612101,5sleep21603888-21perf11:55:002
15814902101,4sleep21581537-21ntpq11:30:192
15511272101,5sleep01551343-21diskstats11:05:120
15393512101,4sleep11539392-21ntpq10:50:181
15354442101,5sleep21535600-21/usr/sbin/munin10:50:112
15306572101,4sleep11530755-21irqstats10:45:131
15304752101,5sleep01530653-21sed10:45:120
14813412101,5sleep11481390-21ntpq09:55:201
14694632101,4sleep01470682-21memory09:45:170
14546872101,4sleep11454735-21ntpq09:30:191
14420432101,5sleep21442297-21kthreadcore09:20:182
14393402101,4sleep21439584-21users09:15:252
14229582101,4sleep01423005-21ntpq09:00:200
14034192101,4sleep11403542-21diskstats08:45:131
14005962101,4sleep11401833-21memory08:40:171
13464432101,5sleep21348200-21cut07:50:162
13425112100,1sleep00-21swapper/007:45:170
13238232101,4sleep11323842-21apt-get07:30:111
1650894291,4sleep01650951-21ld12:35:180
1635070291,4sleep11635194-21/usr/sbin/munin12:20:201
1630629290,3sleep10-21swapper/112:19:511
1624123291,4sleep21624180-21ntp_states12:10:192
1593590292,3sleep21593609-21apt-get11:45:102
1592011291,4sleep21592059-21ntpq11:40:172
1589809291,4sleep01591599-21tr11:40:150
1588349291,5sleep11588517-21grep11:40:121
1572543294,2sleep21573151-21kthreadcore11:25:142
1562096291,4sleep21563303-21cut11:15:162
1509591291,4sleep01509707-21irqstats10:25:120
1466896291,3sleep21466918-21apt-get09:45:102
1449564291,3sleep11449621-21ntp_states09:25:221
1449564291,3sleep11449621-21ntp_states09:25:211
1444959290,1sleep10-21swapper/109:21:331
1444920290,1sleep00-21swapper/009:21:030
1433796291,3sleep01433849-21ntp_states09:10:220
1418691291,4sleep11418720-21systemctl09:00:011
1412539291,4sleep01412619-21tr08:50:220
1408479291,3sleep21408498-21apt-get08:50:122
1403826291,4sleep01405236-21cut08:45:170
1401834290,2sleep20-21swapper/208:40:182
1392722292,3sleep11392740-21apt-get08:35:121
1387149290,3sleep10-21swapper/108:29:341
1380688291,4sleep01380740-21modprobe08:20:200
1380688291,4sleep01380740-21modprobe08:20:200
1376274291,4sleep11376277-21sshd08:18:001
1373330291,3sleep21373516-21kthreadcore08:15:172
1366245291,3sleep01366264-21apt-get08:10:100
1365078291,4sleep01365143-21cut08:05:210
1361011291,2sleep20-21swapper/208:05:102
1356304291,4sleep01357774-21grep08:00:160
1354115290,3sleep11354165-21pool-udisksd07:55:191
1335061291,4sleep01335689-21grep07:40:160
1334225291,4sleep11334296-21munin-run07:40:011
1334143290,3sleep01334049-31frcode07:39:060
1323615291,3sleep01323297-21apt-get07:30:020
1300363290,1sleep20-21swapper/207:10:112
1649035281,3sleep21650747-21latency12:35:162
1641490281,2sleep11641512-21apt-get12:30:101
1631442281,3sleep21631679-21tr12:20:132
1620067281,2sleep01620086-21apt-get12:10:090
1602817280,1sleep00-21swapper/011:50:170
1566782281,2sleep00-21swapper/011:20:110
1557914281,4sleep21559694-21tr11:10:152
1550676281,4sleep11550677-21sshd11:03:331
1545574281,3sleep01540397-21perf10:59:550
1525752281,3sleep11527447-21kthreadcore10:40:161
1524670280,4sleep10-21swapper/110:40:001
1519427280,1sleep10-21swapper/110:34:551
1519427280,1sleep10-21swapper/110:34:551
1514707281,3sleep00-21swapper/010:30:130
1514707281,3sleep00-21swapper/010:30:130
1498647281,2sleep11498666-21apt-get10:15:111
1498643280,4sleep00-21swapper/010:15:100
1497859280,6sleep10-21swapper/110:11:511
1493658281,3sleep21494018-21kthreadcore10:10:152
1483328281,3sleep01485004-21kthreadcore10:00:150
1472100281,3sleep01472108-21acpi09:50:090
1466890280,3sleep10-21swapper/109:45:051
1455733281,3sleep20-21swapper/209:35:002
1435006281,3sleep01435063-21/usr/sbin/munin09:15:110
1428550281,3sleep11428610-21ntpq09:05:221
1424503281,2sleep01424521-21apt-get09:05:130
1423782280,2sleep10-21swapper/109:02:451
1417808281,3sleep01417921-21ps08:55:230
1413269280,2sleep10-21swapper/108:52:511
1402991281,3sleep01403039-21munin-run08:45:010
1382206281,3sleep01382419-21cstates08:25:120
1371982281,3sleep01372053-21irqstats08:15:150
1349880280,1sleep00-21swapper/007:54:100
1329125281,3sleep21329143-21apt-get07:35:132
1314493281,3sleep10-21swapper/107:20:171
1615919271,2sleep00-21swapper/012:05:160
1605709271,2sleep20-21swapper/211:55:162
1571435270,1sleep20-21swapper/211:21:142
1550566271,3sleep21550569-21sshd11:02:102
1539503271,3sleep01539630-21mailstats10:50:210
1524950270,1sleep20-21swapper/210:40:092
1524490270,1sleep20-21swapper/210:37:452
1509708271,2sleep10-21swapper/110:25:131
1508624270,1sleep00-21swapper/010:21:480
1492514270,1sleep00-21swapper/010:05:370
1482181270,1sleep20-21swapper/209:58:012
1423844270,1sleep20-21swapper/209:03:302
1396624271,2sleep01396713-21ntpq08:35:190
1394136271,2sleep21395904-21kthreadcore08:35:172
1392107270,1sleep10-21swapper/108:30:351
1389286271,2sleep00-21swapper/008:30:180
1360550270,1sleep20-21swapper/208:02:262
1334791271,2sleep20-21swapper/207:40:142
1317407271,3sleep20-21swapper/207:20:262
1304147270,2sleep01304292-21strings07:10:180
1636811261,2sleep00-21swapper/012:25:160
1614854260,1sleep10-21swapper/112:05:091
1614849260,1sleep20-21swapper/212:05:062
1608819260,1sleep00-21swapper/011:57:060
1599438261,2sleep10-21swapper/111:50:141
1583048261,2sleep00-21swapper/011:35:140
1571464260,1sleep10-21swapper/111:21:371
1551344261,2sleep10-21swapper/111:05:131
1530756261,2sleep20-21swapper/210:45:142
1508764260,1sleep10-21swapper/110:23:321
1473700261,2sleep20-21swapper/209:50:162
1472672261,2sleep10-21swapper/109:50:151
1466619260,1sleep20-21swapper/209:44:462
1466593260,1sleep10-21swapper/109:44:251
1462551261,2sleep00-21swapper/009:40:160
1461222260,1sleep20-21swapper/209:38:452
1456781261,2sleep00-21swapper/009:35:150
1449889261,2sleep20-21swapper/209:25:272
1449889261,2sleep20-21swapper/209:25:272
1446065261,2sleep00-21swapper/009:25:170
1446065261,2sleep00-21swapper/009:25:170
1439718260,1sleep10-21swapper/109:17:001
1414274261,2sleep20-21swapper/208:55:162
1381401260,1sleep10-21swapper/108:21:571
1381401260,1sleep10-21swapper/108:21:571
1380488260,2sleep20-21swapper/208:20:182
1380488260,2sleep20-21swapper/208:20:182
1354954260,1sleep20-21swapper/207:57:252
1354804260,1sleep00-21swapper/007:55:310
1345542260,2sleep10-21swapper/107:50:151
1344410260,3sleep10-21swapper/107:46:091
1335690261,1sleep10-21swapper/107:40:171
1322191261,2sleep20-21swapper/207:25:192
1306106261,2sleep00-21swapper/007:15:030
1609777250,1sleep10-21swapper/112:00:121
1609027250,1sleep10-21swapper/111:59:441
1581400250,1sleep10-21swapper/111:30:181
1565882250,1sleep00-21swapper/011:15:260
1560492250,1sleep00-21swapper/011:10:210
1495735250,2sleep00-21swapper/010:10:160
1482858251,2sleep10-21swapper/110:00:121
1433574251,1sleep10-21swapper/109:10:191
1387100251,1sleep20-21swapper/208:28:592
1321529250,1sleep10-21swapper/107:25:181
1625983240,1sleep20-21swapper/212:15:122
1620089240,1sleep10-21swapper/112:10:101
1609050240,1sleep20-21swapper/212:00:012
1593818240,1sleep00-21swapper/011:45:120
1577266240,1sleep00-21swapper/011:30:010
1572273240,1sleep00-21swapper/011:25:120
1571947240,1sleep10-21swapper/111:25:011
1482377240,1sleep20-21swapper/210:00:012
1361400240,1sleep10-21swapper/108:05:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional