You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-11 - 09:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Sat Jul 11, 2026 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
183860621520,8sleep21838584-21perf19:10:002
188859321200,3sleep10-21swapper/121:30:151
186646021100,4sleep10-21swapper/120:25:221
186646021100,4sleep10-21swapper/120:25:211
184186921020,3sleep30-21swapper/319:15:193
139099850,70rtkit-daemon0-21swapper/319:09:493
183755727351,15sleep10-21swapper/119:05:181
18662342650,3sleep30-21swapper/320:25:203
18662342650,3sleep30-21swapper/320:25:193
183821126344,13sleep00-21swapper/019:05:340
19001202610,4chrt3056-21PK-Backend22:00:193
19080932570,5sleep20-21swapper/222:25:092
19483562490,2sleep10-21swapper/100:20:121
183871799411,24cyclictest1844604-21chrt19:25:132
183871799386,19cyclictest0-21swapper/223:03:262
183871799384,20cyclictest0-21swapper/221:04:052
19165892340,2sleep00-21swapper/022:50:000
183871799343,19cyclictest0-21swapper/223:39:312
183871799327,13cyclictest1095-21runrttasks23:12:082
1838717993013,15cyclictest1912319-21awk22:35:182
183871799283,23cyclictest1909952-21cpuspeed_turbos22:30:112
183871099270,11cyclictest436-20systemd-journal00:00:010
19080632260,4chrt3056-21PK-Backend22:25:081
183871799267,17cyclictest1934712-21cat23:40:182
183871799261,23cyclictest1944508-21munin-plugin-st00:09:592
18393122250,4sleep03056-21PK-Backend19:10:150
183871799257,17cyclictest1862148-21expr20:15:112
183871799257,16cyclictest0-21swapper/221:10:012
183871799256,17cyclictest1897693-21expr21:55:122
183871799256,17cyclictest1896138-21awk21:50:162
183871799256,16cyclictest1843795-21threads19:20:232
183871799255,18cyclictest1860527-21expr20:10:162
183871799252,21cyclictest1886927-21tr21:25:132
183871799252,21cyclictest1886927-21tr21:25:132
183871799251,4cyclictest0-21swapper/220:45:122
1838717992512,12cyclictest1858190-21sh20:05:012
1838710992515,7cyclictest436-20systemd-journal19:45:000
1838710992510,14cyclictest1880756-21unin-run21:10:010
19428502240,3sleep10-21swapper/100:05:011
183871799248,14cyclictest1930987-21expr23:30:122
183871799247,15cyclictest1904867-21expr22:15:132
183871799247,15cyclictest1877374-21latency_hist21:00:012
183871799245,17cyclictest1935931-21cat23:45:122
1838717992418,5cyclictest0-21swapper/222:45:192
183871799241,3cyclictest1930362-21munin-run23:30:002
1838717992412,10cyclictest1903069-21tr22:10:112
183871799241,20cyclictest1868863-21apt20:35:102
183871799241,19cyclictest1925429-21apt23:15:132
183871799241,19cyclictest1890208-21/usr/sbin/munin21:35:232
183871799240,4cyclictest1884500-21apt-get21:19:592
139099240,3rtkit-daemon0-21swapper/323:18:223
183872099230,21cyclictest3056-21PK-Backend23:50:483
183871799238,13cyclictest1945415-21tail00:10:152
183871799237,14cyclictest1839385-21kernelversion19:10:152
183871799236,15cyclictest1940395-21taskset23:55:322
183871799236,15cyclictest1847097-21open_inodes19:30:212
183871799236,15cyclictest0-21swapper/222:51:002
183871799231,20cyclictest1914773-21sleep22:45:002
183871599231,3cyclictest1884976-21cnf-update-db21:20:001
1838710992316,5cyclictest1887430-21needreboot21:25:180
1838710992316,5cyclictest1887430-21needreboot21:25:180
1838710992315,6cyclictest883-21dbus-daemon00:20:000
139099230,14rtkit-daemon3056-21PK-Backend21:34:153
1838720992220,1cyclictest3056-21PK-Backend19:37:133
1838720992220,1cyclictest0-21swapper/322:34:503
1838720992220,1cyclictest0-21swapper/321:06:353
183872099220,20cyclictest3056-21PK-Backend22:45:263
183871799226,14cyclictest1906764-21idleruntime22:20:152
183871799225,15cyclictest1901354-21cstates22:05:122
183871799223,3cyclictest0-21swapper/221:33:352
183871799222,18cyclictest0-21swapper/219:16:182
183871799221,4cyclictest1927335-21grep23:20:132
183871799221,4cyclictest1848125-21cpuspeed19:35:112
183871799221,3cyclictest1874899-21processes20:50:192
183871799221,18cyclictest1900291-21python322:00:222
1838715992214,6cyclictest436-20systemd-journal21:19:591
183871599220,20cyclictest3056-21PK-Backend22:34:101
1838710992220,1cyclictest3056-21PK-Backend23:30:030
1838710992220,1cyclictest0-21swapper/022:56:360
1838710992220,1cyclictest0-21swapper/022:40:550
1838710992220,1cyclictest0-21swapper/020:02:100
1838710992220,1cyclictest0-21swapper/019:56:010
1838710992216,4cyclictest1953161-21cron00:35:010
1838710992214,6cyclictest1889954-21cron21:34:590
183871099220,21cyclictest0-21swapper/020:13:330
139099220,16rtkit-daemon3056-21PK-Backend00:33:463
183872099218,11cyclictest1881828-21memory21:10:173
1838720992119,1cyclictest3056-21PK-Backend21:25:293
1838720992119,1cyclictest3056-21PK-Backend21:25:283
1838720992119,1cyclictest3056-21PK-Backend20:15:533
1838720992119,1cyclictest3056-21PK-Backend00:12:393
1838720992119,1cyclictest3056-21PK-Backend00:09:073
1838720992119,1cyclictest0-21swapper/323:56:153
1838720992119,1cyclictest0-21swapper/323:44:343
1838720992119,1cyclictest0-21swapper/322:22:133
1838720992119,1cyclictest0-21swapper/320:30:393
1838720992119,1cyclictest0-21swapper/319:25:233
1838720992118,2cyclictest3056-21PK-Backend21:59:083
183872099210,20cyclictest3056-21PK-Backend21:54:023
183872099210,20cyclictest3056-21PK-Backend20:03:083
183872099210,19cyclictest3056-21PK-Backend23:04:103
183872099210,19cyclictest3056-21PK-Backend19:56:273
183871799216,13cyclictest0-21swapper/200:20:012
183871799215,15cyclictest1852187-21needreboot19:45:192
183871799215,14cyclictest436-20systemd-journal00:39:022
183871799215,14cyclictest1894205-21kernelversion21:45:152
183871799215,14cyclictest1886502-21idleruntime-cro21:25:002
183871799214,15cyclictest1941847-21cut00:00:182
183871799214,15cyclictest1922633-21needreboot23:05:172
183871799213,3cyclictest0-21swapper/223:51:012
183871799213,16cyclictest0-21swapper/219:56:282
183871799212,16cyclictest1871460-21python320:40:212
183871799212,16cyclictest1850658-21python319:40:182
183871799211,5cyclictest1892666-21awk21:40:162
183871799211,3cyclictest1880716-21apt-get21:10:032
183871799210,4cyclictest1859404-21awk20:05:222
183871599217,12cyclictest1937326-21unin-run23:50:001
1838715992119,1cyclictest0-21swapper/120:00:181
1838715992115,4cyclictest2156-21gnome-shell00:14:081
183871599210,3cyclictest1317-21systemd20:17:141
183871599210,20cyclictest0-21swapper/123:06:141
183871599210,20cyclictest0-21swapper/122:09:291
183871599210,20cyclictest0-21swapper/100:25:501
183871599210,19cyclictest3056-21PK-Backend23:36:341
183871599210,19cyclictest3056-21PK-Backend19:16:441
1838710992119,1cyclictest0-21swapper/023:09:350
1838710992119,1cyclictest0-21swapper/022:37:170
1838710992119,1cyclictest0-21swapper/022:33:380
1838710992119,1cyclictest0-21swapper/022:09:320
1838710992119,1cyclictest0-21swapper/021:40:530
1838710992119,1cyclictest0-21swapper/021:04:560
1838710992119,1cyclictest0-21swapper/000:26:330
1838710992119,1cyclictest0-21swapper/000:14:240
1838710992116,3cyclictest1869010-21grep20:35:120
1838710992114,5cyclictest1918036-21systemd-udevd22:52:470
1838710992111,7cyclictest1863810-21grep20:20:120
183871099210,20cyclictest0-21swapper/021:12:410
183871099210,20cyclictest0-21swapper/020:45:160
183871099210,20cyclictest0-21swapper/020:07:110
183871099210,20cyclictest0-21swapper/019:36:350
183871099210,20cyclictest0-21swapper/019:24:450
183871099210,20cyclictest0-21swapper/000:37:080
183871099210,20cyclictest0-21swapper/000:20:520
139099210,15rtkit-daemon3056-21PK-Backend00:37:253
1838720992018,1cyclictest3056-21PK-Backend22:50:383
1838720992018,1cyclictest3056-21PK-Backend00:21:513
1838720992018,1cyclictest3056-21PK-Backend00:01:433
1838720992018,1cyclictest0-21swapper/323:21:283
1838720992018,1cyclictest0-21swapper/322:18:353
1838720992018,1cyclictest0-21swapper/320:51:433
1838720992018,1cyclictest0-21swapper/319:12:533
1838720992017,2cyclictest3056-21PK-Backend19:46:193
1838720992013,5cyclictest1878263-21ntp_states21:00:183
1838720992012,6cyclictest1935825-21awk23:45:103
183872099200,19cyclictest0-21swapper/323:32:363
183872099200,18cyclictest3056-21PK-Backend20:22:563
183871799204,14cyclictest0-21swapper/220:31:482
183871799203,3cyclictest0-21swapper/220:26:522
183871799203,3cyclictest0-21swapper/220:26:522
183871799203,2cyclictest0-21swapper/200:29:342
183871799203,15cyclictest0-21swapper/200:20:012
183871799201,5cyclictest1840602-21munin-node00:30:102
183871799201,3cyclictest978-21snmpd22:59:202
183871799200,3cyclictest1853669-21ls19:50:152
1838715992018,1cyclictest0-21swapper/122:57:081
1838715992018,1cyclictest0-21swapper/121:47:381
1838715992018,1cyclictest0-21swapper/100:06:021
1838715992017,2cyclictest3056-21PK-Backend19:12:321
1838715992013,5cyclictest1914789-21pluginstate22:45:001
183871599200,19cyclictest3056-21PK-Backend23:59:131
183871599200,19cyclictest3056-21PK-Backend21:11:091
183871599200,19cyclictest0-21swapper/122:53:491
183871599200,19cyclictest0-21swapper/122:46:391
183871599200,19cyclictest0-21swapper/122:16:111
183871599200,19cyclictest0-21swapper/121:55:371
183871599200,19cyclictest0-21swapper/121:43:051
183871599200,19cyclictest0-21swapper/121:05:071
183871599200,18cyclictest3056-21PK-Backend19:26:451
1838710992019,1cyclictest0-21swapper/019:16:410
1838710992018,1cyclictest0-21swapper/023:39:020
1838710992018,1cyclictest0-21swapper/022:13:110
1838710992018,1cyclictest0-21swapper/022:03:360
1838710992018,1cyclictest0-21swapper/021:38:020
1838710992018,1cyclictest0-21swapper/020:55:430
1838710992018,1cyclictest0-21swapper/020:30:280
1838710992018,1cyclictest0-21swapper/020:28:120
1838710992018,1cyclictest0-21swapper/020:28:120
1838710992018,1cyclictest0-21swapper/000:00:310
1838710992013,5cyclictest1884536-21systemctl21:20:000
1838710992013,4cyclictest1897046-21latency_hist21:55:000
1838710992013,4cyclictest13-21ksoftirqd/000:05:090
1838710992012,6cyclictest1937734-21grep23:50:110
183871099201,17cyclictest1854732-21munin-run19:55:010
183871099200,19cyclictest0-21swapper/019:28:370
139099200,14rtkit-daemon3056-21PK-Backend22:57:193
1838720991917,1cyclictest3056-21PK-Backend22:14:233
1838720991917,1cyclictest3056-21PK-Backend21:37:113
1838720991917,1cyclictest3056-21PK-Backend20:05:043
1838720991917,1cyclictest3056-21PK-Backend19:52:143
1838720991917,1cyclictest3056-21PK-Backend00:19:413
1838720991917,1cyclictest0-21swapper/323:39:403
1838720991917,1cyclictest0-21swapper/300:25:223
1838720991914,3cyclictest1908452-21df22:25:103
1838720991910,6cyclictest42-21ksoftirqd/320:35:103
183872099190,6cyclictest1847273-21chrt19:30:223
183872099190,1cyclictest3056-21PK-Backend21:22:043
183872099190,1cyclictest0-21swapper/320:56:153
183871799190,3cyclictest1864055-21hddtemp_smartct20:20:152
1838715991917,1cyclictest3056-21PK-Backend23:14:311
1838715991917,1cyclictest3056-21PK-Backend21:00:481
1838715991917,1cyclictest0-21swapper/122:12:321
1838715991911,6cyclictest883-21dbus-daemon22:00:021
1838715991911,6cyclictest1938321-21awk23:50:191
183871599190,1cyclictest3056-21PK-Backend23:26:151
183871599190,1cyclictest3056-21PK-Backend22:39:301
183871599190,1cyclictest0-21swapper/121:39:071
183871599190,18cyclictest3056-21PK-Backend20:56:291
183871599190,18cyclictest0-21swapper/121:50:291
183871599190,18cyclictest0-21swapper/120:05:531
183871599190,18cyclictest0-21swapper/119:54:091
183871099192,13cyclictest1851508-21/usr/sbin/munin19:45:250
1838710991917,1cyclictest0-21swapper/023:01:500
1838710991917,1cyclictest0-21swapper/020:54:350
1838710991916,2cyclictest3056-21PK-Backend22:17:220
183871099190,1cyclictest0-21swapper/023:13:270
183871099190,18cyclictest0-21swapper/023:16:440
183871099190,18cyclictest0-21swapper/020:17:250
18675652180,3sleep10-21swapper/120:30:161
1838720991816,1cyclictest3056-21PK-Backend23:05:333
1838720991816,1cyclictest3056-21PK-Backend21:15:293
183872099181,5cyclictest1929896-21cat23:25:193
1838720991810,6cyclictest1912293-21ntpq22:35:183
1838720991810,6cyclictest1872448-21sort20:45:133
1838715991816,1cyclictest0-21swapper/119:41:191
1838715991811,5cyclictest1920376-21expr23:00:131
1838715991811,5cyclictest1887516-21open_files21:25:201
1838715991811,5cyclictest1887516-21open_files21:25:191
1838715991811,5cyclictest1852990-21cron19:49:591
183871099189,7cyclictest1886280-21sshd21:22:250
1838710991816,1cyclictest0-21swapper/022:21:000
183871099181,16cyclictest1847798-21perf19:35:000
1838710991810,6cyclictest1871686-21sshd20:40:250
183872099172,12cyclictest1923696-21apt23:10:123
1838720991715,1cyclictest3056-21PK-Backend22:43:303
1838720991715,1cyclictest0-21swapper/320:13:563
1838720991715,1cyclictest0-21swapper/319:20:583
1838720991713,3cyclictest1095-21runrttasks20:44:063
183871599179,6cyclictest1951688-21find00:30:121
1838715991715,1cyclictest3056-21PK-Backend23:24:531
1838715991715,1cyclictest3056-21PK-Backend23:19:581
1838715991715,1cyclictest3056-21PK-Backend20:10:191
1838715991715,1cyclictest3056-21PK-Backend19:22:181
1838715991712,4cyclictest26-21ksoftirqd/120:35:121
1838715991712,3cyclictest1947570-21uptime00:15:221
1838715991710,5cyclictest1856543-21ls20:00:011
183871099178,6cyclictest1898837-21sshd21:59:250
1838710991715,1cyclictest0-21swapper/022:29:250
1838710991710,6cyclictest1895313-21sed21:50:010
183871599169,5cyclictest1863772-21cat20:20:111
183871599168,6cyclictest1953441-21apt00:35:111
1838715991614,1cyclictest3056-21PK-Backend23:42:501
1838715991614,1cyclictest3056-21PK-Backend20:47:561
1838715991614,1cyclictest0-21swapper/119:38:351
183871599161,13cyclictest1930778-21tr23:30:121
183871099169,5cyclictest883-21dbus-daemon23:25:000
183871099168,6cyclictest1936475-21grep23:45:160
183871099165,9cyclictest1929566-21needreboot23:25:150
1838710991614,1cyclictest0-21swapper/023:42:320
1838720991512,2cyclictest3056-21PK-Backend19:41:193
183871599153,8cyclictest1870596-21apt20:40:121
1838720991410,3cyclictest3056-21PK-Backend22:08:223
183871599140,6cyclictest3056-21PK-Backend19:30:201
183872099139,3cyclictest3056-21PK-Backend21:45:243
1838720991310,2cyclictest3056-21PK-Backend21:40:023
183871599137,4cyclictest1874201-21tr20:50:101
1838715991310,2cyclictest3056-21PK-Backend22:24:361
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional