You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-09 - 19:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Thu Jul 09, 2026 12:45:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
182473699771,68cyclictest1928512-21rm12:35:022
1824727997470,2cyclictest0-21swapper/011:50:010
1824727997470,2cyclictest0-21swapper/008:50:030
1824736997129,38cyclictest1867076-21sh10:25:442
182473699706,51cyclictest1854776-21ssh10:00:012
182472799701,38cyclictest131rcu_preempt10:00:140
182472799701,38cyclictest131rcu_preempt10:00:130
1824736996521,20cyclictest30-21ksoftirqd/212:00:022
1824736996517,42cyclictest1-21systemd08:25:002
182473699651,13cyclictest1874938-21ssh10:42:322
182473699642,16cyclictest1861751-21ssh10:15:012
182473699642,13cyclictest1842832-21munin-run09:35:002
182473699641,15cyclictest1892414-21sshd11:19:252
1824727996455,7cyclictest12-21ksoftirqd/012:37:370
182472799644,52cyclictest1883211-21latency_hist11:00:020
1824727996438,21cyclictest1890326-21latency_hist11:15:000
1824736996321,37cyclictest1830904-21diskmemload11:29:132
1824727996325,17cyclictest18916861ssh11:17:410
1824727996310,6cyclictest131rcu_preempt11:40:020
182473699611,16cyclictest1890348-21munin-run11:15:012
182473699611,15cyclictest48-21kcompactd011:43:102
182473699611,15cyclictest1830887-21cron09:09:592
182473699611,13cyclictest1858510-21ssh10:07:562
1824727996129,16cyclictest1878181-21ssh10:49:220
1824727996129,15cyclictest1853050-21ssh09:56:190
1824727996128,29cyclictest16950irq/224-800000009:20:020
1824727996128,29cyclictest16950irq/224-800000009:20:010
182473699602,16cyclictest1909438-21systemd-run11:55:012
182473699602,16cyclictest1902209-21munin-run11:40:012
182473699602,16cyclictest1838566-21ssh09:25:432
182473699602,14cyclictest1826752-21munin-run07:50:012
1824736996018,37cyclictest30-21ksoftirqd/209:49:032
1824736996018,37cyclictest30-21ksoftirqd/209:49:022
1824736996013,8cyclictest1926137-21ls12:30:022
182473699601,16cyclictest1877620-21ssh10:48:022
1824727996032,13cyclictest1894945-21ssh11:24:470
1824727996032,13cyclictest1894945-21ssh11:24:470
1824727996015,40cyclictest1861782-21systemctl10:15:000
1824727996012,23cyclictest1840447-21kworker/0:109:35:020
1824727996012,23cyclictest1840447-21kworker/0:109:35:020
182473699593,20cyclictest1930900-21sendmail-msp12:40:012
182473699592,18cyclictest1836465-21ssh09:21:282
182473699592,14cyclictest1869282-21sshd10:30:242
182473699592,14cyclictest1869282-21sshd10:30:242
1824736995919,35cyclictest1825946-21sshd07:33:242
1824736995918,36cyclictest1844867-21sh09:39:212
1824736995918,36cyclictest1844867-21sh09:39:212
182473699591,16cyclictest1881730-21ssh10:56:462
182473699591,15cyclictest1847381-21ssh09:44:382
1824727995927,15cyclictest1859383-21ssh10:10:000
1824727995918,36cyclictest12-21ksoftirqd/009:30:020
182473699583,12cyclictest1825820-21wc07:30:022
182473699582,12cyclictest1885549-21munin-run11:05:012
182473699581,15cyclictest1830090-21sshd08:51:232
182473699581,14cyclictest1865009-21sshd10:21:242
182473699581,14cyclictest1865009-21sshd10:21:232
182473699581,13cyclictest1862713-21ssh10:16:432
1824727995835,18cyclictest1830904-21diskmemload09:10:000
1824727995825,28cyclictest309-21in:imuxsock11:00:470
182472799582,49cyclictest1-21systemd09:35:000
1824741995741,11cyclictest1-21systemd12:40:013
1824736995718,36cyclictest30-21ksoftirqd/211:30:422
1824736995716,36cyclictest1835664-21rm09:20:022
1824736995716,36cyclictest1835664-21rm09:20:012
182473699571,14cyclictest1920965-21ssh12:19:142
182473699571,14cyclictest1850420-21ssh09:50:352
182473699571,14cyclictest1850420-21ssh09:50:352
1824727995715,38cyclictest12-21ksoftirqd/012:20:330
182473699562,15cyclictest1905598-21ssh11:46:462
182473699562,13cyclictest1921648-21sh12:20:202
182473699562,13cyclictest1885801-21ssh11:05:072
1824736995615,36cyclictest3467602-21systemd-journal12:05:012
182473699561,12cyclictest1855166-21sh10:00:452
182473699561,12cyclictest1855166-21sh10:00:452
1824727995636,14cyclictest284-21systemd-logind09:50:010
1824727995636,14cyclictest284-21systemd-logind09:50:010
1824727995635,15cyclictest3467602-21systemd-journal09:05:010
1824727995627,14cyclictest1917477-21ssh12:11:380
1824727995627,14cyclictest1897669-21ssh11:30:030
1824727995626,14cyclictest1862927-21ssh10:17:090
1824727995618,34cyclictest151rcuc/012:05:010
182474199552,4cyclictest131rcu_preempt10:45:003
182473699552,14cyclictest1831583-21sh09:11:112
182473699551,14cyclictest1917293-21ssh12:11:152
1824736995511,3cyclictest0-21swapper/209:00:272
182473699551,12cyclictest1827905-21latency_hist08:10:022
182473699551,12cyclictest1827905-21latency_hist08:10:012
1824736995510,7cyclictest1826986-21sshd07:54:252
1824736995510,7cyclictest1826986-21sshd07:54:242
1824732995532,17cyclictest275-21dbus-daemon12:00:011
182472799554,44cyclictest3467602-21systemd-journal12:00:010
1824727995537,13cyclictest1-21systemd09:45:000
1824727995527,13cyclictest1925447-21ssh12:28:300
1824727995527,12cyclictest1850178-21ssh09:50:050
1824727995527,12cyclictest1850178-21ssh09:50:040
1824727995522,29cyclictest12-21ksoftirqd/010:42:000
182473699542,14cyclictest1914635-21ssh12:05:332
182473699542,13cyclictest48-21kcompactd010:35:162
182473699542,13cyclictest48-21kcompactd010:35:152
1824736995412,11cyclictest275-21dbus-daemon08:05:002
182473699541,13cyclictest1893592-21ssh11:21:342
182473699541,13cyclictest1893592-21ssh11:21:332
182473699541,13cyclictest1880509-21ssh10:54:182
1824727995427,12cyclictest1831864-21ssh09:11:460
1824727995425,13cyclictest1867835-21ssh10:27:320
1824736995319,30cyclictest1830429-21sed09:00:012
1824727995330,18cyclictest3467602-21systemd-journal08:30:000
1824727995316,9cyclictest12-21ksoftirqd/011:44:420
182472799531,37cyclictest1887306-21ssh11:08:350
182474199521,5cyclictest0-21swapper/309:45:313
182474199521,5cyclictest0-21swapper/309:45:313
182473699522,15cyclictest1829795-21sort08:45:032
1824736995219,29cyclictest1828841-21systemctl08:30:012
1824736995213,33cyclictest0-21swapper/208:13:242
1824732995246,4cyclictest23-21ksoftirqd/112:05:021
1824727995224,13cyclictest1880119-21ssh10:53:280
1824727995219,30cyclictest12-21ksoftirqd/011:28:590
1824727995215,32cyclictest1872059-21sshd10:36:250
1824727995215,32cyclictest1872059-21sshd10:36:250
182473699512,15cyclictest1829880-21munin-run08:50:022
182473699512,15cyclictest1829880-21munin-run08:50:012
182473299514,3cyclictest131rcu_preempt10:40:001
182473299514,3cyclictest131rcu_preempt10:40:001
182474199505,3cyclictest131rcu_preempt11:09:543
182474199504,3cyclictest131rcu_preempt11:15:213
182474199503,4cyclictest131rcu_preempt09:39:203
182474199503,4cyclictest131rcu_preempt09:39:193
182474199503,3cyclictest131rcu_preempt11:54:523
182474199503,3cyclictest131rcu_preempt07:45:023
182474199502,4cyclictest131rcu_preempt10:02:173
182474199502,4cyclictest131rcu_preempt10:02:173
182473699507,10cyclictest275-21dbus-daemon07:15:002
1824736995011,4cyclictest0-21swapper/208:40:012
1824732995034,12cyclictest3467602-21systemd-journal10:00:001
1824727995043,5cyclictest19350irq/34-mmc110:25:030
1824727995043,5cyclictest19350irq/34-mmc110:25:030
1824727995036,10cyclictest1919971-21cron12:17:010
1824727995029,16cyclictest1838005-21ssh09:24:590
182472799501,23cyclictest0-21swapper/007:20:010
1824727995011,13cyclictest12-21ksoftirqd/008:40:000
182474199494,3cyclictest131rcu_preempt12:15:013
182474199493,4cyclictest131rcu_preempt10:00:013
182474199493,3cyclictest131rcu_preempt11:41:333
182474199493,3cyclictest131rcu_preempt10:47:203
182474199493,3cyclictest131rcu_preempt10:19:513
182474199492,3cyclictest131rcu_preempt07:40:013
182473699492,13cyclictest1827235-21sshd07:59:242
1824736994915,4cyclictest1825165-21journal-offline07:18:592
182473299494,4cyclictest131rcu_preempt11:48:481
182473299494,4cyclictest131rcu_preempt09:55:001
182473299494,4cyclictest131rcu_preempt09:54:591
182473299494,4cyclictest131rcu_preempt08:35:021
182473299494,4cyclictest131rcu_preempt08:35:011
182473299491,3cyclictest131rcu_preempt11:55:001
1824727994925,18cyclictest3467602-21systemd-journal08:14:590
1824727994910,6cyclictest131rcu_preempt12:31:400
182474199486,6cyclictest131rcu_preempt08:55:003
182474199485,3cyclictest131rcu_preempt07:20:003
182474199483,4cyclictest131rcu_preempt11:22:583
182474199483,4cyclictest131rcu_preempt11:22:573
182474199483,3cyclictest131rcu_preempt12:33:463
182474199483,3cyclictest131rcu_preempt11:11:013
182474199483,3cyclictest131rcu_preempt09:28:263
1824741994831,13cyclictest275-21dbus-daemon12:05:013
182474199481,5cyclictest131rcu_preempt12:23:153
182474199481,4cyclictest131rcu_preempt09:42:463
182473299486,8cyclictest23-21ksoftirqd/108:50:021
182473299486,8cyclictest23-21ksoftirqd/108:50:011
182473299484,3cyclictest131rcu_preempt11:19:481
1824732994835,9cyclictest1902208-21systemctl11:40:011
182473299483,3cyclictest131rcu_preempt12:24:041
182473299483,3cyclictest131rcu_preempt07:20:001
182472799483,9cyclictest1871371-21cat10:35:020
182472799483,9cyclictest1871371-21cat10:35:010
182474199474,4cyclictest131rcu_preempt11:01:513
182474199474,4cyclictest131rcu_preempt10:54:483
182474199474,3cyclictest131rcu_preempt12:06:143
182474199474,3cyclictest131rcu_preempt11:35:223
182474199474,3cyclictest131rcu_preempt10:07:063
182474199473,4cyclictest131rcu_preempt09:53:363
182474199473,4cyclictest131rcu_preempt09:53:353
182474199473,4cyclictest131rcu_preempt09:10:243
182474199473,3cyclictest131rcu_preempt12:27:233
182474199473,3cyclictest131rcu_preempt10:33:413
182474199473,3cyclictest131rcu_preempt10:33:413
182474199473,3cyclictest131rcu_preempt10:12:003
1824741994731,12cyclictest275-21dbus-daemon09:00:003
182474199472,3cyclictest131rcu_preempt11:55:223
182474199472,3cyclictest131rcu_preempt11:46:263
1824741994720,5cyclictest37-21ksoftirqd/312:19:593
182473699471,12cyclictest1826199-21sshd07:38:222
182473299477,9cyclictest0-21swapper/110:19:421
182473299477,23cyclictest0-21swapper/111:21:541
182473299477,23cyclictest0-21swapper/111:21:541
182473299474,3cyclictest131rcu_preempt07:34:591
182473299473,4cyclictest131rcu_preempt11:30:261
182473299473,3cyclictest131rcu_preempt10:58:531
182473299473,3cyclictest131rcu_preempt09:36:531
182473299473,3cyclictest131rcu_preempt09:36:521
182473299472,3cyclictest131rcu_preempt09:10:031
182472799479,9cyclictest12-21ksoftirqd/011:51:290
1824727994729,13cyclictest3467602-21systemd-journal07:35:220
182474199465,3cyclictest131rcu_preempt10:20:053
182474199465,3cyclictest131rcu_preempt10:20:043
182474199463,3cyclictest131rcu_preempt11:28:033
182474199463,3cyclictest131rcu_preempt10:25:183
182474199462,4cyclictest131rcu_preempt09:31:043
182474199462,3cyclictest131rcu_preempt09:15:473
182474199462,3cyclictest131rcu_preempt09:15:473
182473699467,19cyclictest1823883-21kworker/u8:207:25:022
182473699463,11cyclictest1829184-21grep08:35:022
182473699463,11cyclictest1829184-21grep08:35:012
182473699461,13cyclictest1828267-21modprobe08:17:002
182473299465,3cyclictest131rcu_preempt10:04:091
182473299465,3cyclictest131rcu_preempt10:04:081
182473299464,3cyclictest131rcu_preempt11:40:581
182473299464,3cyclictest131rcu_preempt11:07:501
182473299463,3cyclictest131rcu_preempt12:09:541
182473299463,3cyclictest131rcu_preempt10:32:101
182473299463,3cyclictest131rcu_preempt10:32:091
182473299462,4cyclictest131rcu_preempt12:40:011
182473299462,3cyclictest131rcu_preempt09:16:541
182473299462,3cyclictest131rcu_preempt09:16:531
182473299461,4cyclictest131rcu_preempt10:51:181
182473299461,3cyclictest131rcu_preempt09:30:071
182473299461,3cyclictest131rcu_preempt09:25:011
1824727994631,10cyclictest309-21in:imuxsock08:35:010
1824727994631,10cyclictest309-21in:imuxsock08:35:000
1824727994627,11cyclictest12-21ksoftirqd/012:08:050
182474199455,4cyclictest131rcu_preempt10:57:113
182474199454,3cyclictest131rcu_preempt08:29:593
182474199453,3cyclictest131rcu_preempt10:36:433
182474199453,3cyclictest131rcu_preempt10:36:423
182474199453,3cyclictest131rcu_preempt09:20:303
182474199453,3cyclictest131rcu_preempt07:35:003
1824741994526,14cyclictest275-21dbus-daemon08:45:013
182474199452,4cyclictest131rcu_preempt08:00:013
182474199452,3cyclictest131rcu_preempt11:33:373
182473299454,4cyclictest131rcu_preempt09:10:001
182473299454,3cyclictest131rcu_preempt10:44:541
182473299454,3cyclictest131rcu_preempt09:28:401
182473299453,4cyclictest131rcu_preempt11:00:131
182473299453,3cyclictest131rcu_preempt12:26:501
182473299452,4cyclictest131rcu_preempt11:10:351
182473299452,3cyclictest131rcu_preempt11:25:141
182473299452,3cyclictest131rcu_preempt10:45:541
182473299452,3cyclictest131rcu_preempt10:14:301
182473299452,3cyclictest131rcu_preempt08:00:001
182473299451,3cyclictest131rcu_preempt12:32:101
182473299451,3cyclictest131rcu_preempt12:15:291
182473299451,3cyclictest131rcu_preempt09:45:161
182473299451,3cyclictest131rcu_preempt09:45:151
182473299451,3cyclictest131rcu_preempt09:40:111
182472799454,9cyclictest12-21ksoftirqd/008:45:000
1824727994538,4cyclictest16950irq/224-800000007:55:250
182474199444,3cyclictest131rcu_preempt07:54:253
182474199444,3cyclictest131rcu_preempt07:54:243
182474199442,4cyclictest131rcu_preempt08:05:003
182474199442,3cyclictest131rcu_preempt08:35:013
182474199442,3cyclictest131rcu_preempt08:35:013
182473299444,3cyclictest131rcu_preempt10:28:001
1824732994429,11cyclictest1-21systemd08:30:011
182473299442,3cyclictest131rcu_preempt12:10:191
182473299441,4cyclictest131rcu_preempt10:20:401
182473299441,4cyclictest131rcu_preempt10:20:401
1824727994430,8cyclictest309-21in:imuxsock07:35:010
1824727994429,11cyclictest1830379-21systemctl08:59:590
182474199434,3cyclictest131rcu_preempt09:09:253
182474199434,3cyclictest131rcu_preempt08:40:013
182474199433,3cyclictest131rcu_preempt08:50:013
182474199433,3cyclictest131rcu_preempt08:50:003
1824741994327,12cyclictest275-21dbus-daemon08:10:013
1824741994327,12cyclictest275-21dbus-daemon08:10:003
182473299436,6cyclictest131rcu_preempt08:55:011
182473299433,3cyclictest131rcu_preempt08:05:011
182473299433,3cyclictest131rcu_preempt07:10:011
1824732994329,10cyclictest1826252-21systemd-run07:40:011
182473299432,3cyclictest131rcu_preempt07:40:231
182473299431,4cyclictest131rcu_preempt10:09:201
1824727994326,10cyclictest12-21ksoftirqd/008:20:010
1824741994229,9cyclictest1-21systemd08:20:003
1824732994220,18cyclictest221rcuc/109:00:011
1824727994235,4cyclictest16950irq/224-800000007:15:000
1824741994120,15cyclictest275-21dbus-daemon07:30:013
1824732994128,9cyclictest3467602-21systemd-journal08:40:001
1824732994125,13cyclictest1-21systemd08:20:001
182473299412,3cyclictest131rcu_preempt08:40:221
182473299412,3cyclictest131rcu_preempt08:25:021
182472799417,26cyclictest1824731-21latency_hist07:10:020
1824727994131,8cyclictest12-21ksoftirqd/007:25:010
182474199403,3cyclictest0-21swapper/308:15:013
1824741994025,11cyclictest309-21in:imuxsock08:20:033
1824736994013,7cyclictest30-21ksoftirqd/207:40:362
182473299402,3cyclictest131rcu_preempt07:49:231
182473299401,4cyclictest131rcu_preempt08:10:001
182473299401,4cyclictest131rcu_preempt08:10:001
1824727994026,10cyclictest12-21ksoftirqd/007:45:010
1824727994024,12cyclictest12-21ksoftirqd/008:10:010
1824727994024,12cyclictest12-21ksoftirqd/008:10:010
182472799401,10cyclictest1827731-21latency_hist08:05:020
182474199392,3cyclictest131rcu_preempt07:25:033
182473299391,4cyclictest0-21swapper/107:15:001
1824727993923,12cyclictest3467602-21systemd-journal08:25:010
182474199385,3cyclictest131rcu_preempt07:15:033
1824732993820,13cyclictest3467602-21systemd-journal08:15:011
182473299381,3cyclictest131rcu_preempt07:30:021
1824727993810,24cyclictest3467602-21systemd-journal08:45:190
1824727993810,24cyclictest3467602-21systemd-journal08:45:180
182474199374,4cyclictest131rcu_preempt07:50:003
182474199374,3cyclictest131rcu_preempt09:03:233
182472799373,8cyclictest131rcu_preempt07:50:000
1824727993725,9cyclictest1827191-21cat07:55:030
1824727993725,9cyclictest1827191-21cat07:55:020
182473699362,13cyclictest1824773-21wc07:10:012
1824732993625,9cyclictest23-21ksoftirqd/107:51:241
1824732993625,9cyclictest23-21ksoftirqd/107:51:241
182473299352,3cyclictest131rcu_preempt07:24:241
1824727993526,6cyclictest1825712-21latency_hist07:30:020
182473299334,3cyclictest131rcu_preempt09:04:231
1824741993124,4cyclictest1824663-21latency_hist07:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional