You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-12 - 16:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Sat Sep 12, 2026 12:45:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1764751998728,55cyclictest1823362-21ssh11:03:080
176475199872,81cyclictest1815737-21ssh10:46:350
1764751998429,51cyclictest1820501-21ssh10:56:510
1764751998429,51cyclictest1820501-21ssh10:56:500
1764751998418,42cyclictest16850irq/134-800000010:07:130
1764754998224,16cyclictest1849156-21cut12:00:021
1764751998218,13cyclictest1784057-21ssh09:37:490
1764751998030,46cyclictest1860182-21ssh12:24:230
1764751998028,32cyclictest1782099-21ssh09:33:460
176475199801,74cyclictest1833396-21ssh11:25:200
1764751998013,6cyclictest16850irq/134-800000010:43:190
1764751997830,38cyclictest1835610-21ssh11:30:040
1764751997818,16cyclictest1809027-21ssh10:32:080
1764754997720,17cyclictest1818052-21ssh10:51:281
176475199773,6cyclictest1844498-21munin-run11:50:010
176475199773,6cyclictest1844498-21munin-run11:50:010
1764751997712,60cyclictest16850irq/134-800000009:41:350
1764751997712,60cyclictest16850irq/134-800000009:41:350
1764751997628,32cyclictest1828296-21ssh11:14:200
1764751997628,32cyclictest1828296-21ssh11:14:200
1764751997627,31cyclictest1802680-21ssh10:18:180
1764751997627,31cyclictest1802680-21ssh10:18:180
1764751997620,51cyclictest1841136-21sshd11:42:250
176475199761,70cyclictest0-21swapper/011:20:170
1764751997611,59cyclictest16850irq/134-800000010:23:480
176475199751,57cyclictest1849721-21ssh12:00:560
176475199751,2cyclictest0-21swapper/009:24:470
1764754997428,15cyclictest3467602-21systemd-journal09:20:301
1764751997428,30cyclictest1861470-21ssh12:27:130
1764751997427,43cyclictest1811039-21ssh10:36:170
1764751997427,43cyclictest1765564-21cat07:25:020
176475199741,69cyclictest0-21swapper/009:12:400
1764751997416,6cyclictest16850irq/134-800000010:10:410
1764751997416,53cyclictest16850irq/134-800000009:57:300
1764751997414,12cyclictest1838461-21ssh11:36:310
1764751997413,56cyclictest16850irq/134-800000009:52:220
1764751997413,56cyclictest16850irq/134-800000009:52:210
1764754997322,15cyclictest1864306-21ssh12:33:361
1764751997330,39cyclictest1766278-21basename07:40:000
1764751997327,41cyclictest1818910-21ssh10:53:310
1764754997230,13cyclictest1800588-21ssh10:13:571
176475199721,11cyclictest1767301-21cut08:00:000
1764754997130,14cyclictest1787592-21ssh09:45:141
176475199718,57cyclictest16850irq/134-800000012:05:190
1764751997124,31cyclictest1830646-21ssh11:19:240
1764751997116,14cyclictest1825873-21ssh11:08:580
1764757997017,13cyclictest3467602-21systemd-journal10:00:002
1764757997015,14cyclictest275-21dbus-daemon10:30:012
1764757997015,14cyclictest275-21dbus-daemon10:30:012
1764754997023,16cyclictest1833258-21cut11:25:011
1764751997012,48cyclictest16850irq/134-800000009:45:310
1764754996913,23cyclictest1787087-21ssh09:44:391
1764754996913,23cyclictest1787087-21ssh09:44:391
1764751996926,39cyclictest1774100-21ssh09:16:060
1764751996925,40cyclictest1845299-21ssh11:51:270
1764751996913,51cyclictest16850irq/134-800000012:37:360
1764754996820,15cyclictest1823639-21ssh11:04:001
1764754996819,20cyclictest1856989-21ssh12:17:151
1764751996816,9cyclictest1854987-21ssh12:12:540
1764757996726,5cyclictest1831414-21sh11:20:462
1764757996725,8cyclictest30-21ksoftirqd/209:20:002
1764754996719,14cyclictest1770898-21cat09:05:021
1764751996722,41cyclictest1848553-21ssh11:58:500
1764757996619,13cyclictest3467602-21systemd-journal07:20:002
1764754996616,14cyclictest1855578-21ssh12:14:261
1764751996612,49cyclictest16850irq/134-800000012:32:530
1764757996523,7cyclictest30-21ksoftirqd/211:55:502
1764757996516,13cyclictest1867177-21cron12:40:002
1764754996520,14cyclictest1765648-21date07:25:021
1764754996512,22cyclictest1814356-21ssh10:43:451
1764751996527,33cyclictest309-21in:imuxsock08:20:000
176475199652,16cyclictest1769143-21sshd08:32:230
1764751996519,13cyclictest1807702-21ssh10:29:280
1764751996519,13cyclictest1807702-21ssh10:29:280
1764757996425,5cyclictest1845574-21sh11:52:052
1764757996423,5cyclictest1852777-21sh12:08:022
1764754996419,11cyclictest1783444-21ssh09:36:221
1764754996417,16cyclictest1843030-21ssh11:46:271
1764754996417,16cyclictest1843030-21ssh11:46:261
176475199648,51cyclictest16850irq/134-800000009:26:360
1764751996422,38cyclictest0-21swapper/012:19:080
176475199642,13cyclictest1766789-21munin-run07:50:000
1764757996313,11cyclictest1787298-21grep09:45:022
1764757996313,11cyclictest1787298-21grep09:45:012
1764757996229,29cyclictest1794761-21sh10:00:492
1764757996225,4cyclictest1853963-21sh12:10:172
1764757996225,12cyclictest30-21ksoftirqd/209:27:572
1764754996249,10cyclictest1803879-21ssh10:20:481
1764754996239,18cyclictest1820526-21ssh10:56:531
1764754996239,18cyclictest1820526-21ssh10:56:531
1764754996224,8cyclictest1806328-21ssh10:26:061
1764754996224,8cyclictest1806328-21ssh10:26:051
1764754996224,6cyclictest1773714-21ssh09:15:181
1764754996220,12cyclictest1839805-21ssh11:39:501
1764751996229,30cyclictest1770280-21latency_hist08:55:020
1764751996218,6cyclictest19350irq/34-mmc109:00:000
176475199621,57cyclictest0-21swapper/009:05:010
176475799614,14cyclictest1817198-21ssh10:50:012
1764757996122,8cyclictest30-21ksoftirqd/212:33:512
1764757996117,15cyclictest3467602-21systemd-journal09:10:002
1764757996117,15cyclictest3467602-21systemd-journal09:10:002
1764754996147,12cyclictest1842108-21ssh11:44:481
1764754996122,12cyclictest1771462-21ssh09:10:301
1764754996115,12cyclictest1792914-21ssh09:57:031
1764751996137,18cyclictest1-21systemd07:20:000
176476199605,11cyclictest37-21ksoftirqd/308:00:003
176475799604,18cyclictest1830802-21cron11:20:002
1764757996020,10cyclictest30-21ksoftirqd/211:08:072
1764754996047,10cyclictest1858935-21ssh12:21:171
1764754996045,11cyclictest1851823-21ssh12:05:301
1764754996043,13cyclictest1817177-21ssh10:49:581
1764754996043,13cyclictest1803048-21ssh10:19:201
1764754996043,13cyclictest1803048-21ssh10:19:201
1764754996039,17cyclictest1810882-21ssh10:35:581
1764754996018,13cyclictest1826321-21ssh11:09:551
176475199603,16cyclictest1796472-21munin-plugin-st10:05:000
1764751996024,32cyclictest12-21ksoftirqd/008:27:160
176475799595,11cyclictest0-21swapper/210:35:002
176475799594,6cyclictest131rcu_preempt12:20:002
1764757995923,5cyclictest1784351-21sh09:38:412
1764754995943,13cyclictest1844713-21ssh11:50:031
1764754995943,13cyclictest1826887-21ssh11:10:511
1764754995943,13cyclictest1826887-21ssh11:10:501
1764754995942,13cyclictest1834086-21ssh11:26:571
1764754995941,14cyclictest1865547-21ssh12:36:011
1764754995938,17cyclictest1808791-21ssh10:31:261
1764754995920,10cyclictest0-21swapper/112:00:231
1764754995916,13cyclictest1829116-21ssh11:15:491
176475799588,13cyclictest1-21systemd08:35:002
1764757995827,9cyclictest30-21ksoftirqd/211:35:092
1764754995845,10cyclictest1835654-21ssh11:30:101
1764754995841,14cyclictest1780685-21ssh09:30:211
1764754995837,16cyclictest1860981-21ssh12:25:531
1764754995820,12cyclictest1778530-21ssh09:25:401
1764754995820,10cyclictest0-21swapper/110:08:351
1764751995821,6cyclictest4163988-1kworker/0:2H07:15:020
1764751995821,6cyclictest4163988-1kworker/0:2H07:15:020
176475199581,53cyclictest0-21swapper/008:50:020
176476199575,12cyclictest37-21ksoftirqd/309:10:033
176476199575,12cyclictest37-21ksoftirqd/309:10:023
1764761995711,6cyclictest131rcu_preempt10:18:063
1764761995711,6cyclictest131rcu_preempt10:18:053
1764761995711,6cyclictest131rcu_preempt10:04:563
1764757995723,4cyclictest1812233-21sh10:39:132
1764757995722,8cyclictest30-21ksoftirqd/209:33:172
1764757995721,6cyclictest30-21ksoftirqd/207:15:022
1764757995721,6cyclictest30-21ksoftirqd/207:15:022
1764754995745,9cyclictest1789885-21ssh09:50:101
1764754995745,9cyclictest1789885-21ssh09:50:101
1764754995742,12cyclictest1794980-21ssh10:01:231
1764754995741,11cyclictest1766296-21munin-run07:40:011
1764754995738,15cyclictest1769167-21munin-run08:35:001
1764754995723,7cyclictest1767275-21sshd07:59:241
176475199572,16cyclictest3467602-21systemd-journal07:55:000
1764757995626,4cyclictest1840461-21sh11:40:552
1764754995640,12cyclictest1767019-21sshd07:53:251
176476199553,15cyclictest37-21ksoftirqd/311:40:003
1764757995546,5cyclictest1756002-21kworker/2:107:40:022
1764757995535,15cyclictest3467602-21systemd-journal09:15:012
1764757995526,9cyclictest30-21ksoftirqd/212:27:472
1764757995524,5cyclictest1814016-21sh10:42:492
1764757995523,7cyclictest30-21ksoftirqd/210:08:182
1764754995542,10cyclictest1768301-21cron08:17:001
176475799545,8cyclictest30-21ksoftirqd/208:36:112
1764757995446,5cyclictest1770214-21kworker/2:209:25:012
1764757995424,4cyclictest1834593-21sh11:28:122
1764757995422,5cyclictest1791106-21sh09:53:072
1764757995422,5cyclictest1791106-21sh09:53:072
1764754995440,11cyclictest1768898-21sshd08:29:241
1764754995439,10cyclictest1767822-21munin-run08:10:011
1764754995439,10cyclictest1765710-21modprobe07:25:021
1764754995438,12cyclictest1768577-21sshd08:20:241
1764754995433,16cyclictest1769408-21cron08:40:001
176475199543,17cyclictest1768076-21munin-plugin-st08:15:000
176475199543,17cyclictest1768076-21munin-plugin-st08:15:000
1764751995424,26cyclictest1766066-21latency_hist07:35:010
176475199541,2cyclictest0-21swapper/009:10:020
176475199541,2cyclictest0-21swapper/009:10:020
176476199531,9cyclictest37-21ksoftirqd/312:38:523
1764754995340,10cyclictest1768087-21munin-run08:15:011
1764754995340,10cyclictest1768087-21munin-run08:15:011
1764754995339,10cyclictest1770435-21sshd08:58:241
1764754995334,15cyclictest364-21snmpd08:43:211
1764751995320,28cyclictest1-21systemd08:05:010
176475199531,48cyclictest0-21swapper/008:45:020
176476199528,8cyclictest361rcuc/307:25:013
176476199526,8cyclictest37-21ksoftirqd/309:50:003
176476199525,8cyclictest37-21ksoftirqd/311:35:023
176476199525,13cyclictest37-21ksoftirqd/311:15:433
176476199523,7cyclictest37-21ksoftirqd/312:00:283
1764761995211,5cyclictest131rcu_preempt09:18:443
1764761995210,5cyclictest131rcu_preempt10:48:583
1764757995244,5cyclictest1770214-21kworker/2:210:12:042
1764757995243,5cyclictest1769194-21kworker/2:310:21:142
1764757995242,6cyclictest1769194-21kworker/2:310:18:002
1764757995242,6cyclictest1769194-21kworker/2:310:18:002
1764754995240,8cyclictest3467602-21systemd-journal07:43:091
1764754995237,11cyclictest364-21snmpd08:04:041
176475199522,17cyclictest1769454-21true08:40:020
176476199517,7cyclictest37-21ksoftirqd/309:10:243
176476199516,10cyclictest37-21ksoftirqd/309:40:013
176476199515,8cyclictest37-21ksoftirqd/310:20:403
1764761995142,6cyclictest361rcuc/312:17:443
1764761995142,6cyclictest361rcuc/309:50:303
1764761995142,6cyclictest361rcuc/309:50:303
1764761995134,13cyclictest275-21dbus-daemon11:25:003
1764761995110,5cyclictest131rcu_preempt10:10:003
1764757995143,5cyclictest1770214-21kworker/2:212:22:132
176475799513,18cyclictest1821826-21modprobe11:00:002
176475799513,18cyclictest1821826-21modprobe11:00:002
176476199505,12cyclictest37-21ksoftirqd/310:40:313
1764761995014,13cyclictest37-21ksoftirqd/311:05:013
1764757995041,6cyclictest1770214-21kworker/2:212:04:472
1764754995034,11cyclictest1764944-21cron07:15:001
1764754995034,11cyclictest1764944-21cron07:15:001
1764751995010,11cyclictest1767789-21sshd08:05:300
176476199497,11cyclictest37-21ksoftirqd/310:31:243
176476199495,7cyclictest37-21ksoftirqd/312:00:013
1764761994940,7cyclictest361rcuc/311:26:473
176476199493,9cyclictest1845259-21rm11:51:223
1764761994930,14cyclictest3467602-21systemd-journal09:30:013
1764761994915,5cyclictest131rcu_preempt09:57:353
176475799495,5cyclictest131rcu_preempt08:30:032
1764757994941,5cyclictest1770214-21kworker/2:211:47:432
1764757994941,5cyclictest1770214-21kworker/2:211:47:422
1764757994941,5cyclictest1770214-21kworker/2:211:01:552
1764757994941,5cyclictest1769194-21kworker/2:310:54:572
1764754994937,9cyclictest3467602-21systemd-journal08:45:171
1764754994935,9cyclictest1765989-21cron07:35:001
176476199488,4cyclictest131rcu_preempt12:25:583
1764761994836,8cyclictest361rcuc/309:33:053
176476199483,10cyclictest37-21ksoftirqd/310:53:513
176476199482,5cyclictest131rcu_preempt11:44:563
1764761994817,6cyclictest131rcu_preempt10:15:013
1764761994817,19cyclictest37-21ksoftirqd/309:24:153
1764761994812,6cyclictest131rcu_preempt12:30:143
1764757994839,5cyclictest1530674-21kworker/2:308:10:022
176476199478,8cyclictest37-21ksoftirqd/312:08:063
176476199477,10cyclictest37-21ksoftirqd/312:13:393
176476199476,10cyclictest37-21ksoftirqd/311:47:043
176476199476,10cyclictest37-21ksoftirqd/311:47:043
176476199474,12cyclictest37-21ksoftirqd/310:57:213
176476199474,12cyclictest37-21ksoftirqd/310:57:213
176476199473,7cyclictest37-21ksoftirqd/310:26:353
176476199473,7cyclictest37-21ksoftirqd/310:26:343
1764761994735,9cyclictest37-21ksoftirqd/312:20:543
176476199473,11cyclictest37-21ksoftirqd/311:15:023
176476199473,11cyclictest37-21ksoftirqd/311:15:023
176476199473,10cyclictest37-21ksoftirqd/310:37:013
1764757994739,5cyclictest1770214-21kworker/2:211:13:502
1764757994739,5cyclictest1770214-21kworker/2:211:13:502
1764754994736,8cyclictest3467602-21systemd-journal07:48:281
1764754994735,9cyclictest364-21snmpd09:09:201
1764754994735,9cyclictest364-21snmpd09:09:201
1764754994732,11cyclictest48-21kcompactd007:16:011
1764754994717,25cyclictest1770377-21latency_hist08:55:021
1764761994636,7cyclictest37-21ksoftirqd/308:25:013
1764761994614,12cyclictest37-21ksoftirqd/309:44:563
1764761994614,12cyclictest37-21ksoftirqd/309:44:553
1764757994613,6cyclictest131rcu_preempt11:32:362
176476199455,7cyclictest37-21ksoftirqd/307:45:013
1764761994536,6cyclictest37-21ksoftirqd/311:08:493
1764761994532,10cyclictest37-21ksoftirqd/307:30:023
1764751994511,8cyclictest12-21ksoftirqd/008:25:020
176476199446,8cyclictest37-21ksoftirqd/308:47:543
1764757994436,5cyclictest1530674-21kworker/2:308:00:012
1764757994427,13cyclictest0-21swapper/209:05:022
1764751994413,7cyclictest1765786-21sed07:30:020
176475799439,5cyclictest131rcu_preempt09:48:592
176475799434,9cyclictest30-21ksoftirqd/208:14:182
176475799434,9cyclictest30-21ksoftirqd/208:14:182
176475799432,5cyclictest131rcu_preempt09:00:002
176476199427,9cyclictest37-21ksoftirqd/308:35:013
1764761994223,14cyclictest275-21dbus-daemon09:00:013
176476199421,15cyclictest0-21swapper/307:54:543
176475799424,6cyclictest131rcu_preempt08:25:002
1764757994227,11cyclictest1769947-21cron08:50:002
176475799418,5cyclictest131rcu_preempt07:31:582
1764757994118,16cyclictest0-21swapper/207:10:022
176476199408,5cyclictest131rcu_preempt08:40:003
1764761994026,11cyclictest3467602-21systemd-journal07:40:013
1764761994022,13cyclictest275-21dbus-daemon07:20:003
176475799405,8cyclictest30-21ksoftirqd/207:27:272
176475199401,4cyclictest0-21swapper/007:43:190
1764757993926,9cyclictest30-21ksoftirqd/208:45:002
1764757993919,15cyclictest0-21swapper/208:20:002
176476199387,9cyclictest37-21ksoftirqd/309:01:383
1764761993829,7cyclictest37-21ksoftirqd/307:35:003
1764761993828,5cyclictest1768059-21cron08:15:003
1764761993828,5cyclictest1768059-21cron08:15:003
1764757993824,11cyclictest30-21ksoftirqd/207:25:012
1764761993725,8cyclictest37-21ksoftirqd/308:55:013
176476199371,7cyclictest0-21swapper/307:13:473
176476199371,7cyclictest0-21swapper/307:13:473
176476199371,4cyclictest131rcu_preempt08:19:093
1764757993728,6cyclictest30-21ksoftirqd/207:45:022
1764757993725,9cyclictest309-21in:imuxsock07:51:242
1764757993724,9cyclictest3467602-21systemd-journal07:50:002
1764754993725,9cyclictest0-21swapper/107:10:021
1764751993726,8cyclictest12-21ksoftirqd/007:10:020
1764761993623,9cyclictest3467602-21systemd-journal08:01:243
176476199361,6cyclictest0-21swapper/308:40:263
176476199361,6cyclictest0-21swapper/308:26:253
176476199361,16cyclictest0-21swapper/307:45:283
1764757993621,11cyclictest309-21in:imuxsock08:00:292
1764761993525,8cyclictest37-21ksoftirqd/308:05:243
1764761993424,7cyclictest37-21ksoftirqd/307:10:013
1764757993018,9cyclictest1770186-21sshd08:54:242
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional