You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-04 - 06:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sat Jul 04, 2026 00:43:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1673745996348,6cyclictest0-21swapper/219:25:362
1673745996348,6cyclictest0-21swapper/219:25:362
1673745996145,10cyclictest1000-21mta-sts-daemon20:45:382
1673745996145,10cyclictest1000-21mta-sts-daemon20:45:382
1673743995339,10cyclictest1798391-21cron23:35:010
1673743995130,15cyclictest12650irq/32-eth000:10:380
1673744995041,6cyclictest1764839-21idleruntime-cro22:40:011
1673744994839,6cyclictest1799148-21if_wlan023:35:281
1673744994838,5cyclictest0-21swapper/122:25:401
1673745994737,4cyclictest0-21swapper/221:35:372
1673745994733,10cyclictest1762584-21kernelversion22:35:312
1673743994638,4cyclictest1829897-21missed_timers00:25:320
1673743994638,4cyclictest1735526-21proc_pri21:50:420
167374599454,35cyclictest1735385-21ssh21:50:372
1673745994522,4cyclictest0-21swapper/222:25:372
1673743994539,3cyclictest0-21swapper/021:20:010
1673745994435,5cyclictest1000-21mta-sts-daemon23:05:362
1673744994434,6cyclictest0-21swapper/119:15:011
1673743994435,6cyclictest4113014-21cups-browsed21:38:000
1673745994334,6cyclictest265-21systemd-journal23:00:002
1673744994332,8cyclictest1744804-21mailstats22:05:461
1673743994336,4cyclictest1000-21mta-sts-daemon21:00:350
167374399432,32cyclictest1744514-21ssh22:05:370
1673744994232,6cyclictest421-21dbus-daemon22:55:021
1673744994232,6cyclictest421-21dbus-daemon22:55:021
1673744994229,9cyclictest1805978-21timerandwakeup23:45:461
1673745994130,8cyclictest1835945-21missed_timers00:35:312
1673745994127,11cyclictest351ktimers/222:00:352
167374599412,30cyclictest1833089-21ssh00:30:352
1673744994133,5cyclictest1752629-21sendmail22:20:011
1673744994133,4cyclictest0-21swapper/121:45:461
1673744994131,7cyclictest1813736-21cups-browsed00:13:111
1673745994032,5cyclictest1813680-21systemctl00:00:002
1673745994032,5cyclictest1810656-21latency_hist23:55:012
1673745994026,5cyclictest171rcu_preempt21:30:392
167374599401,33cyclictest1732256-21ssh21:45:322
1673744994033,3cyclictest0-21swapper/123:00:001
1673744994032,5cyclictest1830231-21sh00:25:421
1673744994031,6cyclictest1704254-21fschecks_time20:50:291
1673743994033,4cyclictest0-21swapper/023:40:340
1673745993931,5cyclictest265-21systemd-journal23:50:012
1673744993931,4cyclictest0-21swapper/119:20:011
1673744993928,7cyclictest1000-21mta-sts-daemon23:13:261
167374399393,28cyclictest1771966-21rm22:50:330
167374399393,28cyclictest1771966-21rm22:50:330
1673743993931,4cyclictest1737940-21sort21:55:270
1673743993931,4cyclictest1737940-21sort21:55:270
1673743993930,7cyclictest1669752-21kworker/u8:12+xprtiod22:45:340
167374599383,29cyclictest4113010-21cupsd23:35:362
1673745993830,4cyclictest0-21swapper/222:45:242
1673745993829,6cyclictest421-21dbus-daemon22:05:302
1673745993826,9cyclictest1728733-21cpu21:40:262
167374499384,27cyclictest1747695-21ssh22:10:421
1673744993829,6cyclictest1743470-21cron22:05:001
1673744993825,9cyclictest485-21in:imuxsock21:44:281
1673744993811,14cyclictest28-21ksoftirqd/100:15:341
1673743993830,5cyclictest420-21avahi-daemon23:50:280
167374399381,32cyclictest1805583-21ssh23:45:360
167374599374,5cyclictest171rcu_preempt22:30:392
1673745993727,7cyclictest265-21systemd-journal00:27:372
1673745993726,8cyclictest1793154-21latency23:25:292
1673744993729,5cyclictest1755688-21cron22:25:011
1673744993727,7cyclictest420-21avahi-daemon22:30:151
1673743993731,4cyclictest1793006-21ssh23:25:280
1673743993729,4cyclictest1003-21mta-sts-daemon23:21:350
1673745993629,4cyclictest1698255-21fschecks_time20:30:272
1673745993629,3cyclictest0-21swapper/222:18:092
1673745993629,3cyclictest0-21swapper/200:25:012
1673745993626,6cyclictest0-21swapper/221:10:002
167374499365,10cyclictest28-21ksoftirqd/122:45:381
1673744993630,3cyclictest0-21swapper/120:00:021
1673744993626,3cyclictest171rcu_preempt19:45:401
1673743993628,5cyclictest1787818-21unixbench_multi23:15:530
1673743993627,5cyclictest420-21avahi-daemon22:44:450
1673743993627,5cyclictest151ktimers/019:10:350
1673743993626,7cyclictest1752981-21/usr/sbin/munin22:20:470
1673743993624,10cyclictest1818041-21wc00:05:440
1673743993622,11cyclictest1676722-21cron19:20:010
167374399361,29cyclictest1775162-21ssh22:55:390
1673745993528,4cyclictest1715857-21munin-run21:20:012
1673745993528,4cyclictest0-21swapper/219:50:012
1673745993526,6cyclictest1687155-21head19:50:462
1673745993525,4cyclictest0-21swapper/222:40:352
1673744993527,5cyclictest1735905-21unixbench_multi21:50:471
1673744993527,4cyclictest1771056-21kworker/u8:13-rpciod23:32:061
1673744993526,6cyclictest1-21systemd21:00:011
1673744993526,4cyclictest0-21swapper/100:30:371
1673744993524,8cyclictest1804526-21cut23:45:021
1673744993520,10cyclictest1767710-21sshd22:44:281
1673744993515,3cyclictest171rcu_preempt23:55:581
1673743993529,3cyclictest0-21swapper/023:55:460
1673743993519,10cyclictest151ktimers/019:50:330
1673743993517,13cyclictest151ktimers/021:30:350
1673745993420,8cyclictest0-21swapper/223:10:432
1673744993425,6cyclictest1815061-21mailstats00:00:441
1673744993424,6cyclictest4113014-21cups-browsed21:21:411
167374499340,10cyclictest28-21ksoftirqd/120:45:351
167374499340,10cyclictest28-21ksoftirqd/120:45:351
1673743993427,4cyclictest1757010-21mailstats22:25:440
1673743993427,4cyclictest1709730-21grep21:10:000
1673743993425,6cyclictest4113014-21cups-browsed22:32:270
1673743993425,6cyclictest1813738-21kworker/u8:12-xprtiod00:30:410
1673743993424,7cyclictest1826617-21idleruntime00:20:270
1673743993422,9cyclictest1703621-21tail20:45:460
1673743993422,9cyclictest1703621-21tail20:45:460
1673743993421,10cyclictest1814094-21sh00:00:240
1673745993327,3cyclictest0-21swapper/223:20:012
1673745993326,4cyclictest1706728-21awk21:00:002
1673745993326,3cyclictest0-21swapper/222:10:342
1673745993325,5cyclictest1778323-21processes23:00:412
1673745993325,5cyclictest1695269-21fschecks_time20:20:272
167374499338,10cyclictest28-21ksoftirqd/123:50:381
167374499337,8cyclictest28-21ksoftirqd/119:10:011
1673744993326,4cyclictest261rcuc/120:00:411
1673744993326,4cyclictest1795343-21cron23:30:011
1673744993324,6cyclictest1710843-21ntp_kernel_pll_21:10:421
1673744993324,6cyclictest1710843-21ntp_kernel_pll_21:10:421
1673744993322,8cyclictest1683915-21switchtime19:40:461
1673744993322,8cyclictest1680602-21ntp_kernel_pll_19:30:361
1673744993322,8cyclictest1680602-21ntp_kernel_pll_19:30:361
1673744993320,3cyclictest171rcu_preempt21:30:361
1673744993312,2cyclictest171rcu_preempt21:05:441
167374399335,20cyclictest1778139-21ssh23:00:360
167374399335,17cyclictest12650irq/32-eth023:10:410
1673743993328,3cyclictest1755711-21kworker/u8:16-xprtiod22:35:240
1673743993326,4cyclictest1687744-21fschecks_count19:55:270
1673743993324,5cyclictest151ktimers/021:10:410
1673743993324,5cyclictest151ktimers/021:10:410
1673743993321,9cyclictest1740977-21cpu22:00:250
167374399331,26cyclictest665-21snmpd19:30:420
167374399331,26cyclictest665-21snmpd19:30:420
1673745993225,4cyclictest1000-21mta-sts-daemon20:38:242
1673745993224,5cyclictest4113014-21cups-browsed21:55:472
1673745993224,5cyclictest4113014-21cups-browsed21:55:472
1673745993223,5cyclictest1003-21mta-sts-daemon23:22:322
167374599322,26cyclictest1814765-21ssh00:00:352
1673745993221,7cyclictest351ktimers/220:15:352
167374499329,8cyclictest28-21ksoftirqd/123:05:351
167374499326,8cyclictest28-21ksoftirqd/121:17:271
1673744993225,3cyclictest0-21swapper/121:40:011
1673744993222,5cyclictest0-21swapper/100:36:411
1673744993217,3cyclictest171rcu_preempt20:35:591
1673743993225,4cyclictest1825905-21munin-run00:20:000
1673743993225,4cyclictest1699242-21systemd-run20:35:010
1673743993225,3cyclictest0-21swapper/023:05:470
1673743993224,6cyclictest1716853-21needreboot21:20:360
1673743993223,6cyclictest4113014-21cups-browsed21:48:270
1673743993221,9cyclictest1694716-21cron20:20:010
1673743993216,5cyclictest1801451-21systemctl23:40:010
167374599313,22cyclictest1753689-21ssh22:20:342
1673745993124,4cyclictest1693214-21cron20:15:012
1673745993123,5cyclictest1818405-21unixbench_multi00:05:492
1673745993123,5cyclictest1677048-21aten_r2power_en19:20:252
1673745993122,5cyclictest1001-21mta-sts-daemon22:55:032
1673745993122,5cyclictest1001-21mta-sts-daemon22:55:032
167374499318,9cyclictest28-21ksoftirqd/100:20:391
167374499314,11cyclictest28-21ksoftirqd/120:30:381
167374499313,16cyclictest1001-21mta-sts-daemon19:50:391
1673744993125,3cyclictest0-21swapper/121:55:351
1673744993125,3cyclictest0-21swapper/121:55:351
1673744993123,4cyclictest0-21swapper/120:45:011
1673744993122,5cyclictest0-21swapper/123:15:281
1673744993113,7cyclictest171rcu_preempt20:20:361
1673744993110,4cyclictest171rcu_preempt19:35:331
1673744993110,15cyclictest627-21lldpd23:20:361
167374399314,17cyclictest12650irq/32-eth019:40:400
1673743993126,2cyclictest0-21swapper/000:40:020
1673743993120,8cyclictest1690244-21idleruntime-cro20:05:010
1673745993023,3cyclictest0-21swapper/223:33:062
1673745993023,3cyclictest0-21swapper/220:50:402
1673745993022,5cyclictest1688595-21unixbench_multi19:55:452
1673745993022,5cyclictest0-21swapper/200:20:002
167374599302,18cyclictest1716987-21awk21:20:412
167374499305,8cyclictest28-21ksoftirqd/100:05:331
1673744993021,5cyclictest0-21swapper/121:25:461
1673743993022,5cyclictest1702234-21munin-plugin-st20:45:010
167374399302,20cyclictest1747566-21ssh22:10:360
1673743993020,8cyclictest4113014-21cups-browsed20:24:230
1673743993017,9cyclictest0-21swapper/019:30:020
1673743993017,9cyclictest0-21swapper/019:30:020
1673745992923,3cyclictest0-21swapper/219:15:012
1673745992921,5cyclictest1681105-21unixbench_multi19:30:472
1673745992921,5cyclictest1681105-21unixbench_multi19:30:472
167374499295,17cyclictest428-21mta-sts-daemon20:05:331
1673744992920,5cyclictest271ktimers/120:10:401
1673744992919,7cyclictest1697734-21/usr/sbin/munin20:30:021
1673744992912,3cyclictest171rcu_preempt21:00:341
167374399292,22cyclictest665-21snmpd19:20:400
1673745992821,4cyclictest1711374-21sed21:10:532
1673745992821,4cyclictest1711374-21sed21:10:532
1673745992820,5cyclictest1682725-21cron19:40:002
1673745992818,7cyclictest1684254-21sh19:45:012
1673745992816,9cyclictest1002-21mta-sts-daemon21:28:482
167374499281,9cyclictest28-21ksoftirqd/123:00:361
167374399289,13cyclictest12650irq/32-eth021:25:380
167374399288,16cyclictest4113014-21cups-browsed20:55:370
1673743992822,4cyclictest1682761-21systemd-run19:40:010
1673743992821,4cyclictest1729148-21needreboot21:40:330
1673745992720,4cyclictest1691082-21ntp_kernel_err20:05:322
1673744992715,2cyclictest171rcu_preempt19:25:341
1673744992715,2cyclictest171rcu_preempt19:25:341
1673744992713,7cyclictest171rcu_preempt19:20:291
1673744992711,3cyclictest171rcu_preempt20:15:281
1673743992715,10cyclictest0-21swapper/020:50:470
1673743992713,11cyclictest1699870-21kernelversion20:35:300
167374599262,18cyclictest4113014-21cups-browsed20:40:352
1673745992618,5cyclictest1801810-21/usr/sbin/munin23:40:352
1673745992611,2cyclictest171rcu_preempt00:10:402
167374399256,12cyclictest151ktimers/020:25:380
1673743992519,3cyclictest1685657-21unixbench_singl19:45:460
1673743992519,3cyclictest0-21swapper/022:15:390
167374599242,4cyclictest1707571-21ntpq21:00:342
1673745992417,4cyclictest1676756-21tr19:20:012
1673745992417,4cyclictest1002-21mta-sts-daemon20:02:202
1673743992416,5cyclictest4113014-21cups-browsed20:11:430
1673743992416,5cyclictest4113014-21cups-browsed20:08:250
1673743992210,8cyclictest1673742-21cyclictest19:10:010
167374599217,4cyclictest171rcu_preempt20:25:342
167374699191,9cyclictest0-21swapper/300:25:393
167374699182,8cyclictest0-21swapper/321:45:363
167374699181,6cyclictest0-21swapper/320:40:363
1673745991812,3cyclictest0-21swapper/219:10:022
167374699174,10cyclictest0-21swapper/300:15:343
167374699172,9cyclictest0-21swapper/323:20:373
167374699172,7cyclictest0-21swapper/322:01:273
167374699163,6cyclictest0-21swapper/321:40:393
167374699162,7cyclictest0-21swapper/323:05:373
167374699162,7cyclictest0-21swapper/300:15:013
167374699161,7cyclictest0-21swapper/321:20:383
167374699161,7cyclictest0-21swapper/320:20:393
167374699161,7cyclictest0-21swapper/319:30:373
167374699161,7cyclictest0-21swapper/319:30:373
167374699161,7cyclictest0-21swapper/300:05:403
167374699153,6cyclictest0-21swapper/322:15:393
167374699152,7cyclictest0-21swapper/321:35:353
167374699151,8cyclictest0-21swapper/300:20:403
167374699151,7cyclictest0-21swapper/322:25:383
167374699151,7cyclictest0-21swapper/322:20:353
167374699151,7cyclictest0-21swapper/321:50:343
167374699151,7cyclictest0-21swapper/319:50:363
167374699151,7cyclictest0-21swapper/319:20:363
167374699144,6cyclictest0-21swapper/321:10:413
167374699144,6cyclictest0-21swapper/321:10:413
167374699142,8cyclictest0-21swapper/323:00:333
167374699142,6cyclictest0-21swapper/323:25:263
167374699142,6cyclictest0-21swapper/321:15:343
167374699141,8cyclictest0-21swapper/323:10:413
167374699141,7cyclictest0-21swapper/322:55:363
167374699141,7cyclictest0-21swapper/322:50:363
167374699141,7cyclictest0-21swapper/322:50:363
167374699141,7cyclictest0-21swapper/320:30:353
167374699141,7cyclictest0-21swapper/320:10:373
167374699141,7cyclictest0-21swapper/319:25:343
167374699141,7cyclictest0-21swapper/319:25:343
167374699141,7cyclictest0-21swapper/319:15:393
167374699141,7cyclictest0-21swapper/300:00:343
167374699141,6cyclictest0-21swapper/321:30:343
167374699141,6cyclictest0-21swapper/321:00:383
167374699141,6cyclictest0-21swapper/320:25:363
167374699141,6cyclictest0-21swapper/300:30:343
167374699134,7cyclictest0-21swapper/300:35:253
167374699133,7cyclictest0-21swapper/322:40:363
167374699132,7cyclictest0-21swapper/322:10:013
167374699132,7cyclictest0-21swapper/319:35:343
167374699132,6cyclictest0-21swapper/321:25:253
167374699132,6cyclictest0-21swapper/300:00:013
167374699131,7cyclictest0-21swapper/323:50:383
167374699131,7cyclictest0-21swapper/323:30:333
167374699131,6cyclictest0-21swapper/320:50:343
167374699131,5cyclictest0-21swapper/322:35:033
167374699130,7cyclictest0-21swapper/320:05:343
167374699123,4cyclictest0-21swapper/319:10:423
167374699122,6cyclictest0-21swapper/322:35:443
167374699122,6cyclictest0-21swapper/322:10:343
167374699122,6cyclictest0-21swapper/322:00:013
167374699122,6cyclictest0-21swapper/322:00:013
167374699121,6cyclictest0-21swapper/323:40:383
167374699121,6cyclictest0-21swapper/320:35:333
167374699112,7cyclictest0-21swapper/322:45:233
167374699111,6cyclictest0-21swapper/323:35:353
167374699111,5cyclictest0-21swapper/323:15:463
167374699111,5cyclictest0-21swapper/320:55:343
167374699111,5cyclictest0-21swapper/319:55:313
167374699110,5cyclictest0-21swapper/321:05:383
167374699101,6cyclictest0-21swapper/320:45:383
167374699101,6cyclictest0-21swapper/320:45:383
16737469991,6cyclictest0-21swapper/323:45:253
16737469991,5cyclictest0-21swapper/319:45:253
16737469991,5cyclictest0-21swapper/319:40:353
16737469991,4cyclictest0-21swapper/320:15:343
16737469991,4cyclictest0-21swapper/320:00:243
16737469981,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional