You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-06 - 21:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Fri Mar 06, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1745055215141,104sleep30-21swapper/307:06:133
174511328655,10sleep10-21swapper/107:06:551
174463328363,12sleep20-21swapper/207:05:232
174034627951,17sleep00-21swapper/007:05:050
174548199251,14cyclictest0-21swapper/312:35:463
174548199222,15cyclictest1893663-21chrt09:20:103
174548199215,12cyclictest1953346-21taskset09:52:543
174548199211,16cyclictest0-21swapper/308:45:353
1745481992110,7cyclictest1936250-21taskset09:43:283
174548199210,17cyclictest2119689-21taskset11:26:553
174548199202,14cyclictest1817203-21turbostat08:15:003
1745481992014,3cyclictest2077309-21taskset11:03:503
174548199201,16cyclictest66950irq/132-enp1s0-TxRx-312:10:103
174548199201,16cyclictest0-21swapper/312:26:343
174548199201,15cyclictest66950irq/132-enp1s0-TxRx-310:19:213
174548199201,15cyclictest0-21swapper/311:13:013
174548199201,15cyclictest0-21swapper/310:56:283
174548199201,15cyclictest0-21swapper/308:31:363
22365592191,13sleep21875761-21diskmemload12:35:052
174548199199,6cyclictest2122385-21taskset11:30:073
1745481991915,2cyclictest0-21swapper/312:05:163
174548199191,15cyclictest0-21swapper/312:32:573
174548199191,15cyclictest0-21swapper/311:15:113
174548199191,15cyclictest0-21swapper/310:31:513
174548199191,15cyclictest0-21swapper/310:28:513
174548199191,14cyclictest0-21swapper/312:18:213
174548199191,14cyclictest0-21swapper/311:36:233
174548199191,14cyclictest0-21swapper/310:53:013
174548199191,14cyclictest0-21swapper/310:45:583
174548199191,14cyclictest0-21swapper/310:45:573
174548199191,14cyclictest0-21swapper/309:57:153
174548199190,17cyclictest0-21swapper/309:45:503
174548199190,16cyclictest1822500-21chrt08:19:273
21844822181,4sleep11875761-21diskmemload12:05:141
21772712188,5sleep22179020-21cut12:00:172
21188942181,3sleep00-21swapper/011:25:520
21030362181,3sleep10-21swapper/111:18:041
174548199189,5cyclictest0-21swapper/309:10:003
174548199181,6cyclictest0-21swapper/310:40:113
174548199181,5cyclictest66950irq/132-enp1s0-TxRx-309:17:093
1745481991811,4cyclictest1927811-21taskset09:39:193
174548199181,14cyclictest0-21swapper/312:20:033
174548199181,14cyclictest0-21swapper/312:03:393
174548199181,14cyclictest0-21swapper/311:50:103
174548199181,14cyclictest0-21swapper/311:43:303
174548199181,14cyclictest0-21swapper/310:06:433
174548199181,14cyclictest0-21swapper/309:30:513
174548199181,14cyclictest0-21swapper/309:26:033
174548199181,14cyclictest0-21swapper/309:10:173
174548199181,13cyclictest2003387-21chrt10:20:203
174548199181,13cyclictest0-21swapper/307:20:113
174548199180,15cyclictest0-21swapper/311:59:583
174548199180,15cyclictest0-21swapper/311:05:103
174548199180,15cyclictest0-21swapper/310:37:553
174548199180,14cyclictest0-21swapper/310:10:073
174548199180,14cyclictest0-21swapper/307:50:113
20951932171,3sleep20-21swapper/211:14:152
20402512171,3sleep20-21swapper/210:41:572
19360582171,3sleep00-21swapper/009:43:140
1745481991714,1cyclictest0-21swapper/311:20:143
1745481991712,3cyclictest0-21swapper/311:49:033
174548199171,13cyclictest0-21swapper/308:35:123
174548199170,14cyclictest0-21swapper/310:00:273
174548199170,14cyclictest0-21swapper/307:35:033
22094192161,4sleep21875761-21diskmemload12:19:142
21732782161,2sleep00-21swapper/011:57:510
21224132161,4sleep12122432-21apt-get11:30:101
21208862161,2sleep10-21swapper/111:28:301
21007042161,4sleep02100788-21find11:15:260
20949352161,4sleep11875761-21diskmemload11:13:541
20865622161,2sleep00-21swapper/011:09:290
20777462161,4sleep11875761-21diskmemload11:04:241
20660992161,9sleep00-21swapper/010:55:380
19810602161,4sleep21875761-21diskmemload10:09:502
19263312161,3sleep00-21swapper/009:37:240
19242732161,4sleep21875761-21diskmemload09:35:212
17571222161,9sleep01757179-21ntpq07:15:200
174548199161,12cyclictest0-21swapper/307:25:173
174548199160,13cyclictest0-21swapper/308:52:033
174548199160,13cyclictest0-21swapper/308:40:193
174548199160,13cyclictest0-21swapper/307:49:093
174548199160,13cyclictest0-21swapper/307:10:203
22359802150,2sleep00-21swapper/012:34:400
21826402151,9sleep00-21swapper/012:03:360
21266052151,3sleep22126648-21ntpq11:30:202
20685702150,9sleep21875761-21diskmemload10:58:532
20331962150,2sleep00-21swapper/010:39:100
20167462151,7sleep12017011-21df10:30:131
19997502151,10sleep22000957-21cut10:20:162
19259422151,7sleep11925953-21sh09:36:531
19168162151,2sleep20-21swapper/209:31:262
18841782150,1sleep20-21swapper/209:14:372
18813222151,2sleep00-21swapper/009:10:500
1745481991513,1cyclictest0-21swapper/307:55:053
174548199151,2cyclictest0-21swapper/308:25:183
1745481991510,2cyclictest0-21swapper/307:40:103
174548199150,2cyclictest0-21swapper/307:15:013
174548199150,13cyclictest0-21swapper/308:20:203
174548199150,12cyclictest0-21swapper/309:00:203
174548199150,12cyclictest0-21swapper/308:55:103
22002812141,4sleep22200292-21ssh12:13:442
21393982141,3sleep00-21swapper/011:39:470
21055322142,8sleep11875761-21diskmemload11:20:161
20308882141,3sleep21875761-21diskmemload10:36:072
19965572140,2sleep00-21swapper/010:17:090
19855892141,7sleep21985741-21ntp_192.168.11410:10:172
19816092141,8sleep01981869-21df_inode10:10:120
19188952140,9sleep11875761-21diskmemload09:34:091
174548199140,3cyclictest0-21swapper/308:00:123
174548199140,12cyclictest0-21swapper/307:30:163
22194932131,6sleep21875761-21diskmemload12:25:142
22153932131,8sleep22215405-21ssh12:20:342
21890442131,6sleep00-21swapper/012:05:290
21805172131,6sleep11875761-21diskmemload12:00:481
21665352131,3sleep12166564-21apt-get11:55:121
20865322132,7sleep11875761-21diskmemload11:09:261
20227362131,8sleep02022750-21ssh10:31:570
20075372131,4sleep22007793-21latency_hist10:25:002
19953562131,8sleep11875761-21diskmemload10:15:341
19617482131,3sleep21961760-21ssh09:57:312
17885762135,4sleep01789405-21cat07:45:170
17588732133,6sleep11759032-21cut07:20:141
17586612131,8sleep01758871-21idleruntime07:20:120
174548199138,3cyclictest1811542-21chrt08:05:553
22271872121,6sleep11875761-21diskmemload12:29:341
22185082120,5sleep00-21swapper/012:24:410
22021772120,8sleep02202284-21sh12:15:140
21928662121,5sleep02192896-21apt-get12:10:100
21841012122,4sleep22184124-21apt-get12:05:112
21577012121,6sleep21875761-21diskmemload11:50:082
21515002121,5sleep12152752-21memory11:45:181
21498912123,5sleep02151499-21cut11:45:170
21454432121,3sleep22145457-21ssh11:41:102
20406352121,5sleep00-21swapper/010:42:260
20246352121,3sleep22024649-21ssh10:34:272
19540822121,7sleep11875761-21diskmemload09:53:531
19005892121,3sleep01900599-21ssh09:23:100
18977822121,8sleep21897838-21rm09:20:182
18709692121,5sleep11871178-21hddtemp_smartct09:05:151
18446932121,7sleep21846249-21cut08:40:152
18020902121,7sleep01803745-21cut08:00:170
17895482121,6sleep21789597-21ntpq07:45:202
17842842121,5sleep21784333-21ntpq07:40:192
22367492111,5sleep12236939-21diskstats12:35:131
22328992111,4sleep11875761-21diskmemload12:30:361
22245832111,4sleep01875761-21diskmemload12:26:090
22154792111,4sleep11875761-21diskmemload12:20:411
21539922111,6sleep22154006-21ssh11:45:542
21451292111,4sleep11875761-21diskmemload11:40:451
21388422111,4sleep11875761-21diskmemload11:39:031
21113762111,3sleep00-21swapper/011:22:320
21014722111,3sleep20-21swapper/211:16:002
20956752111,3sleep00-21swapper/011:14:540
20788872111,6sleep22078995-21perl11:05:142
20614012111,3sleep10-21swapper/110:55:151
20578072111,3sleep00-21swapper/010:51:370
20570102111,4sleep11875761-21diskmemload10:50:341
20410092111,4sleep11875761-21diskmemload10:42:561
20315332111,4sleep11875761-21diskmemload10:36:591
20132432111,2sleep00-21swapper/010:25:580
20056352111,5sleep11875761-21diskmemload10:22:321
19787032111,4sleep11875761-21diskmemload10:06:441
19774952110,2sleep01977726-21ssh10:05:270
19702512111,6sleep11875761-21diskmemload10:02:061
19701572111,5sleep21950830-21kworker/u16:0+rpciod10:01:592
19625862110,5sleep00-21swapper/009:58:370
19430422111,3sleep20-21swapper/209:45:522
19429762111,4sleep11875761-21diskmemload09:45:481
19163862111,3sleep00-21swapper/009:30:510
19026992111,6sleep01902809-21tune2fs09:25:130
18987222111,4sleep11875761-21diskmemload09:20:421
18906542111,2sleep20-21swapper/209:16:382
18760272111,3sleep10-21swapper/109:10:031
18684262111,6sleep21869276-21cat09:00:192
18438102111,5sleep01843882-21munin-run08:40:000
18229132111,5sleep01823116-21diskstats08:20:130
17797752111,5sleep11779690-31find07:38:461
17512792111,6sleep21752722-21if_err_enp1s007:15:012
22430522100,1sleep00-21swapper/012:37:280
22022852101,6sleep12203811-21kthreadcore12:15:161
21935012102,4sleep12194875-21kthreadcore12:10:151
21650462101,2sleep00-21swapper/011:53:340
21464232100,1sleep00-21swapper/011:42:280
21390592101,3sleep20-21swapper/211:39:202
21115572101,3sleep20-21swapper/211:22:452
20762972101,3sleep00-21swapper/011:02:300
20752062100,4sleep20-21swapper/211:01:052
20498522100,1sleep10-21swapper/110:48:011
20498522100,1sleep10-21swapper/110:48:001
20493432101,3sleep21875761-21diskmemload10:47:222
20493432101,3sleep21875761-21diskmemload10:47:212
20078772101,4sleep11875761-21diskmemload10:25:071
20059772101,5sleep02005991-21ssh10:22:590
19960222100,3sleep20-21swapper/210:16:272
19814972101,2sleep11875761-21diskmemload10:10:051
19692212100,1sleep00-21swapper/010:00:450
19622482101,4sleep11875761-21diskmemload09:58:101
19528952101,4sleep01875761-21diskmemload09:52:190
19523192101,3sleep21875761-21diskmemload09:51:342
19418822101,5sleep01941981-21ntpq09:45:190
19367962101,4sleep11875761-21diskmemload09:44:111
19345632101,5sleep21934577-21ssh09:41:162
19088532100,4sleep21875761-21diskmemload09:27:292
19074102101,2sleep10-21swapper/109:25:351
18904762101,2sleep00-21swapper/009:16:240
18898822100,1sleep10-21swapper/109:15:371
18496832101,5sleep11849862-21cut08:45:141
18432172101,4sleep21843434-21users08:35:262
18214102102,4sleep21821523-21ntpq08:15:182
18059222101,4sleep21806166-21users08:00:272
17905602105,2sleep10-21swapper/107:49:591
17797662100,5sleep01779690-31find07:38:410
17744852100,4sleep21774103-21apt-get07:35:022
17740902101,5sleep11774484-21cut07:35:001
17731082100,4sleep21773155-21ntpq07:30:202
17642582101,4sleep11765432-21grep07:25:151
17641882101,4sleep01764257-21irqstats07:25:140
17637602101,4sleep21763976-21diskstats07:25:122
17583852101,5sleep21758601-21ls07:20:112
2228514291,4sleep22229327-21kthreadcore12:30:172
2171469291,2sleep20-21swapper/211:55:292
2162876290,1sleep10-21swapper/111:50:431
2128037290,2sleep00-21swapper/011:31:260
2118605291,4sleep22118616-21ssh11:25:302
2056250291,4sleep22056301-21ntpq10:50:232
2048711291,4sleep02048726-21ssh10:46:320
2048711291,4sleep02048726-21ssh10:46:320
1865377291,3sleep01865396-21apt-get09:00:120
1864110291,3sleep01864157-21ntpq08:55:200
1828656291,4sleep11829339-21cut08:25:151
1806799291,3sleep11806818-21apt-get08:05:111
1864591280,2sleep20-21swapper/208:55:252
1854743280,1sleep10-21swapper/108:50:121
1843114280,4sleep10-21swapper/108:35:241
1822274280,1sleep00-21swapper/008:16:400
1813128281,3sleep11814826-21kthreadcore08:10:151
1812126280,2sleep00-21swapper/008:10:100
1811676281,3sleep21811677-21sshd08:07:402
1795595280,1sleep10-21swapper/107:51:411
1791368281,3sleep01791980-21kthreadcore07:50:150
1750465280,2sleep00-21swapper/007:11:360
1870667270,2sleep00-21swapper/009:05:020
1869460271,2sleep10-21swapper/109:00:221
1860016270,1sleep10-21swapper/108:55:051
1859432270,1sleep20-21swapper/208:50:492
1854721270,2sleep00-21swapper/008:50:110
1854152270,1sleep20-21swapper/208:46:072
1851175271,3sleep00-21swapper/008:45:170
1848927270,1sleep10-21swapper/108:42:271
1837435270,2sleep10-21swapper/108:30:201
1833155270,3sleep21833289-21latency_hist08:30:012
1821884271,3sleep10-21swapper/108:15:251
1803746271,3sleep11805348-21grep08:00:181
1801181270,1sleep10-21swapper/107:59:531
1784981270,1sleep00-21swapper/007:41:260
1870679260,1sleep20-21swapper/209:05:122
1838328260,1sleep00-21swapper/008:33:080
1838209260,1sleep20-21swapper/208:31:362
1831991260,1sleep00-21swapper/008:25:180
1817124260,4sleep20-21swapper/208:14:102
1810704261,2sleep00-21swapper/008:05:180
1797237261,2sleep00-21swapper/007:55:160
1780181260,1sleep10-21swapper/107:40:101
1750742261,2sleep10-21swapper/107:15:011
1746257261,2sleep10-21swapper/107:10:161
1844383250,1sleep00-21swapper/008:40:130
1827232250,1sleep10-21swapper/108:20:251
1826801250,1sleep20-21swapper/208:20:182
1796524250,1sleep20-21swapper/207:55:132
1773558250,1sleep00-21swapper/007:30:260
1790609240,1sleep20-21swapper/207:50:012
1745408240,1sleep20-21swapper/207:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional