You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-24 - 20:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Fri Jul 24, 2026 12:43:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2543480997763,8cyclictest0-21swapper/211:30:532
254348099532,25cyclictest2662272-21ssh11:25:482
254348099532,25cyclictest2662272-21ssh11:25:482
2543480995031,15cyclictest2677975-21unixbench_singl11:50:582
2543478994940,6cyclictest1742083-21cups-browsed11:58:050
254347899492,9cyclictest2665370-21ssh11:30:530
2543480994840,5cyclictest2577996-21systemd-run09:05:012
2543480994839,6cyclictest2574974-21true08:55:012
254347999482,39cyclictest2604828-21ssh09:50:491
2543478994733,10cyclictest0-21swapper/011:05:460
2543480994618,25cyclictest2644539-21timerandwakeup10:55:572
2543480994617,3cyclictest0-21swapper/210:50:502
254348099453,36cyclictest2610904-21ssh10:00:482
254348099453,36cyclictest2610904-21ssh10:00:482
254348099451,19cyclictest2548949-21modprobe07:25:512
2543479994537,5cyclictest2609812-21latency_hist10:00:021
254347899453,8cyclictest171rcu_preempt10:35:420
2543478994529,12cyclictest2668825-21grep11:35:580
2543480994435,6cyclictest1-21systemd08:10:012
254348099441,14cyclictest2701644-21ntpq12:30:492
2543480994410,10cyclictest1742083-21cups-browsed10:32:132
2543479994436,5cyclictest2670263-21systemd-run11:40:011
2543480994333,7cyclictest265-21systemd-journal11:15:012
2543480994319,20cyclictest1742083-21cups-browsed09:16:002
2543480994317,3cyclictest0-21swapper/211:45:502
2543480994315,2cyclictest0-21swapper/212:25:412
2543480994231,6cyclictest0-21swapper/212:15:012
2543480994221,17cyclictest2619261-21aten_r2power_cu10:15:342
2543480994219,20cyclictest2685904-21aten_r2power_po12:05:372
2543480994217,3cyclictest0-21swapper/210:10:452
2543480994215,2cyclictest0-21swapper/211:00:472
254347899423,7cyclictest0-21swapper/012:15:490
254347899422,33cyclictest2647223-21ssh11:00:500
2543480994132,4cyclictest0-21swapper/210:05:512
2543480994132,4cyclictest0-21swapper/210:05:512
2543480994124,13cyclictest0-21swapper/209:35:482
2543480994116,3cyclictest0-21swapper/209:10:562
2543479994134,4cyclictest2688476-21latency_hist12:10:021
254348099404,16cyclictest351ktimers/208:30:472
2543480994018,19cyclictest0-21swapper/210:20:382
2543480994017,2cyclictest0-21swapper/209:55:592
2543480994016,3cyclictest0-21swapper/209:20:472
2543480994016,3cyclictest0-21swapper/209:20:472
2543480994015,5cyclictest2659450-21processes11:20:562
2543480994015,5cyclictest2659450-21processes11:20:562
2543480994014,5cyclictest1003-21mta-sts-daemon12:21:472
2543480994014,3cyclictest0-21swapper/209:40:522
2543480994010,4cyclictest1000-21mta-sts-daemon08:10:512
2543479994032,5cyclictest2702189-21sed12:30:581
2543479994032,5cyclictest2697443-21idleruntime-cro12:25:011
2543479994032,5cyclictest2633919-21run-parts10:40:011
2543479994031,6cyclictest2635209-21proc_pri10:40:511
2543479994031,6cyclictest2635209-21proc_pri10:40:511
2543478994029,4cyclictest1001-21mta-sts-daemon09:11:440
2543478994020,13cyclictest12650irq/32-eth009:40:450
2543478994019,13cyclictest151ktimers/009:30:450
254348099393,9cyclictest2704673-21sshd12:35:492
2543480993916,2cyclictest0-21swapper/210:40:012
2543480993912,5cyclictest1002-21mta-sts-daemon07:10:372
254348099391,15cyclictest2601847-21ntpq09:45:462
2543478993935,2cyclictest0-21swapper/009:15:490
2543480993829,6cyclictest2543499-21latency_hist07:10:022
254348099382,16cyclictest2635079-21ntpq10:40:472
254348099382,16cyclictest2635079-21ntpq10:40:472
254348099382,14cyclictest2652103-21munin-run11:10:022
2543480993817,18cyclictest2655735-21cpuspeed11:15:372
2543480993814,2cyclictest0-21swapper/212:15:362
2543480993813,7cyclictest2669034-21sh11:36:152
2543480993811,6cyclictest0-21swapper/211:40:532
2543480993810,5cyclictest2606412-21sh09:53:572
2543478993831,3cyclictest0-21swapper/012:25:460
2543478993829,5cyclictest0-21swapper/010:05:520
2543478993829,5cyclictest0-21swapper/010:05:520
2543478993822,11cyclictest0-21swapper/009:35:340
254348099373,14cyclictest2683009-21df12:00:382
254348099373,14cyclictest2683009-21df12:00:382
2543480993731,3cyclictest0-21swapper/208:50:022
2543480993729,4cyclictest2591833-21latency_hist09:30:022
2543480993729,4cyclictest2591833-21latency_hist09:30:022
2543480993717,17cyclictest0-21swapper/209:35:022
254348099371,14cyclictest2637889-21memory10:45:412
254348099371,13cyclictest2625764-21latency10:25:442
2543479993728,6cyclictest1-21systemd09:45:011
2543479993728,5cyclictest2679283-21ssh11:55:011
2543479993728,5cyclictest2587223-21unixbench_multi09:20:581
2543479993728,5cyclictest2587223-21unixbench_multi09:20:581
254347899375,25cyclictest151ktimers/007:25:480
2543478993731,3cyclictest2640087-21latency_hist10:50:020
2543478993731,3cyclictest0-21swapper/010:25:480
2543478993730,3cyclictest0-21swapper/007:30:510
2543478993725,5cyclictest0-21swapper/009:50:500
2543478993724,9cyclictest2607642-21latency09:55:480
2543480993615,3cyclictest0-21swapper/211:55:442
2543480993615,2cyclictest0-21swapper/208:05:022
2543479993629,4cyclictest0-21swapper/110:30:521
2543479993628,5cyclictest2699076-21unixbench-2d12:25:581
2543479993628,5cyclictest2618942-21latency_hist10:15:021
2543479993628,4cyclictest0-21swapper/110:15:381
2543479993627,6cyclictest2626551-21grep10:26:011
2543479993626,7cyclictest2649049-21ssh11:05:011
2543479993626,7cyclictest1742083-21cups-browsed09:49:451
254347999362,31cyclictest2589747-21ssh09:25:471
254347999362,31cyclictest2589747-21ssh09:25:471
254347999361,29cyclictest2692606-21sshd12:15:501
254347899369,22cyclictest2616986-21ssh10:10:490
2543478993628,4cyclictest0-21swapper/012:05:540
2543478993627,5cyclictest0-21swapper/012:30:570
2543478993626,8cyclictest2628314-21aten_r2power_cu10:30:360
2543478993622,6cyclictest12750irq/33-eth011:20:470
2543478993622,6cyclictest12750irq/33-eth011:20:470
2543478993621,10cyclictest151ktimers/008:55:530
2543478993620,12cyclictest0-21swapper/010:50:440
254347899361,28cyclictest2671446-21ssh11:40:500
254348099353,8cyclictest2571450-21ntpq08:40:532
2543479993523,9cyclictest2611243-21switchtime10:00:571
2543479993523,9cyclictest2611243-21switchtime10:00:571
254347999352,30cyclictest2641163-21ssh10:50:501
2543478993529,3cyclictest0-21swapper/008:45:010
2543478993528,4cyclictest1002-21mta-sts-daemon12:04:070
2543478993528,4cyclictest1002-21mta-sts-daemon12:04:070
2543478993527,5cyclictest2589624-21ssh09:25:430
2543478993527,5cyclictest2589624-21ssh09:25:430
2543478993527,4cyclictest0-21swapper/011:15:510
2543478993526,5cyclictest1742083-21cups-browsed09:45:090
2543478993526,4cyclictest0-21swapper/012:35:520
2543478993525,7cyclictest2661833-21sh11:25:390
2543478993525,7cyclictest2661833-21sh11:25:390
254347999343,25cyclictest2671478-21ssh11:40:511
2543479993426,5cyclictest2681001-21grep11:55:581
2543479993422,9cyclictest0-21swapper/111:10:441
254347999342,23cyclictest2665340-21apt-get11:30:531
2543478993430,2cyclictest0-21swapper/010:20:010
254348099333,13cyclictest1742083-21cups-browsed07:40:462
2543480993327,3cyclictest0-21swapper/207:20:022
2543480993326,4cyclictest2570465-21munin-plugin-st08:40:012
2543479993327,3cyclictest0-21swapper/109:40:011
2543479993326,4cyclictest1002-21mta-sts-daemon09:30:431
2543479993325,5cyclictest2655574-21aten_r2power_en11:15:361
2543479993325,5cyclictest2613274-21aten_r2power_fr10:05:351
2543479993325,5cyclictest2613274-21aten_r2power_fr10:05:351
2543479993324,6cyclictest421-21dbus-daemon12:00:391
2543479993324,6cyclictest421-21dbus-daemon12:00:391
2543479993324,6cyclictest420-21avahi-daemon10:57:551
2543479993324,6cyclictest0-21swapper/107:20:011
2543479993323,6cyclictest2662160-21needreboot11:25:431
2543479993323,6cyclictest2662160-21needreboot11:25:431
254347899333,23cyclictest2695627-21ssh12:20:510
2543478993325,5cyclictest485-21in:imuxsock07:15:010
254347899332,26cyclictest2689483-21ssh12:10:480
2543480993227,2cyclictest0-21swapper/207:40:012
254347999326,19cyclictest2650224-21ssh11:05:471
2543479993226,3cyclictest0-21swapper/108:05:001
2543479993225,4cyclictest1003-21mta-sts-daemon07:35:361
2543479993224,5cyclictest2582498-21ssh09:15:011
2543479993223,6cyclictest420-21avahi-daemon07:25:431
2543478993223,6cyclictest2634655-21ssh10:40:380
2543478993223,6cyclictest2634655-21ssh10:40:380
2543480993120,7cyclictest341rcuc/207:50:532
2543480993120,7cyclictest341rcuc/207:50:532
2543479993124,4cyclictest1002-21mta-sts-daemon08:40:531
2543479993124,4cyclictest1001-21mta-sts-daemon12:12:241
2543479993123,4cyclictest0-21swapper/110:20:491
254347999312,20cyclictest2638136-21ssh10:45:481
2543478993124,3cyclictest0-21swapper/010:23:260
2543478993120,8cyclictest0-21swapper/008:50:460
2543478993116,9cyclictest151ktimers/011:50:500
2543478993110,17cyclictest2674463-21sshd11:45:500
2543480993024,3cyclictest0-21swapper/208:15:392
254347999303,21cyclictest2566941-21sshd08:25:501
2543479993024,3cyclictest0-21swapper/109:20:011
2543479993023,4cyclictest2676271-21munin-run11:50:011
2543479993023,4cyclictest2553864-21unixbench_multi07:40:571
2543479993021,6cyclictest0-21swapper/108:20:011
2543478993024,4cyclictest2655290-21latency_hist11:15:030
2543478993022,3cyclictest0-21swapper/010:55:500
254347899302,21cyclictest1742083-21cups-browsed09:20:460
254347899302,21cyclictest1742083-21cups-browsed09:20:460
2543478993020,8cyclictest2566498-21fschecks_time08:25:390
2543480992922,3cyclictest0-21swapper/208:55:502
2543480992919,8cyclictest2565286-21needreboot08:20:422
254347999295,16cyclictest428-21mta-sts-daemon07:55:501
254347999295,16cyclictest428-21mta-sts-daemon07:55:501
254347999292,19cyclictest2556410-21sshd07:50:501
254347999292,19cyclictest2556410-21sshd07:50:501
2543479992921,5cyclictest2554499-21fschecks_time07:45:411
2543479992921,5cyclictest2554499-21fschecks_time07:45:411
2543479992921,3cyclictest0-21swapper/107:15:021
2543479992920,6cyclictest2572195-21kworker/u8:11+xprtiod09:05:561
2543478992924,3cyclictest2573466-21grep08:50:010
2543478992919,7cyclictest2546388-21unixbench_singl07:15:590
254348099283,19cyclictest265-21systemd-journal08:25:512
2543480992821,3cyclictest0-21swapper/207:45:542
2543480992821,3cyclictest0-21swapper/207:45:542
2543479992820,5cyclictest2705823-21ssh12:37:521
2543479992820,4cyclictest0-21swapper/108:55:481
2543479992819,5cyclictest0-21swapper/108:35:521
2543479992818,5cyclictest2567987-21fschecks_time08:30:381
254347899286,17cyclictest421-21dbus-daemon07:40:010
2543478992820,5cyclictest2610671-21latency10:00:420
2543478992820,5cyclictest2610671-21latency10:00:420
2543478992819,7cyclictest2556329-21ntp_kernel_pll_07:50:470
2543478992819,7cyclictest2556329-21ntp_kernel_pll_07:50:470
2543478992819,6cyclictest439-21systemd-logind07:10:010
2543478992819,6cyclictest12650irq/32-eth008:40:010
2543480992719,5cyclictest1742083-21cups-browsed07:57:192
2543480992719,5cyclictest1742083-21cups-browsed07:57:192
2543479992721,3cyclictest1002-21mta-sts-daemon09:00:491
2543479992719,5cyclictest2550852-21unixbench_multi07:30:581
2543479992717,7cyclictest0-21swapper/111:20:401
2543479992717,7cyclictest0-21swapper/111:20:401
254347899275,7cyclictest171rcu_preempt07:55:490
254347899275,7cyclictest171rcu_preempt07:55:490
2543478992722,3cyclictest1000-21mta-sts-daemon08:30:480
254347899271,20cyclictest265-21systemd-journal07:20:500
2543478992711,11cyclictest0-21swapper/008:05:400
2543480992617,6cyclictest265-21systemd-journal07:35:012
2543478992618,6cyclictest2577115-21kernelversion09:00:400
2543480992520,3cyclictest0-21swapper/209:10:012
2543480992519,3cyclictest0-21swapper/207:25:012
254347999254,14cyclictest428-21mta-sts-daemon08:45:491
2543479992519,3cyclictest0-21swapper/107:10:021
2543479992518,4cyclictest0-21swapper/108:55:011
2543478992519,4cyclictest1000-21mta-sts-daemon08:20:030
2543478992513,9cyclictest2565896-21unixbench_singl08:20:570
254347999244,13cyclictest428-21mta-sts-daemon08:10:461
2543479992419,2cyclictest0-21swapper/108:10:001
2543479992418,3cyclictest2547967-21cron07:25:011
254347899248,13cyclictest2559960-21cron08:05:000
2543478992417,5cyclictest2579471-21cut09:10:010
2543478992410,11cyclictest0-21swapper/007:45:010
2543479992311,8cyclictest2564752-21/usr/sbin/munin08:20:351
254347899232,17cyclictest2554666-21cat07:45:430
254347899232,17cyclictest2554666-21cat07:45:430
2543478992310,4cyclictest2561481-21kworker/u8:15-xprtiod08:10:360
254348199193,10cyclictest0-21swapper/312:15:503
254348199184,7cyclictest0-21swapper/309:45:523
254348199173,5cyclictest0-21swapper/311:40:473
254348199164,4cyclictest0-21swapper/308:05:463
254348199162,8cyclictest0-21swapper/309:50:533
254348199162,6cyclictest0-21swapper/311:25:383
254348199162,6cyclictest0-21swapper/311:25:383
254348199161,8cyclictest0-21swapper/308:50:513
254348199161,7cyclictest0-21swapper/310:15:463
254348199154,4cyclictest0-21swapper/309:40:473
254348199153,8cyclictest0-21swapper/310:25:363
254348199153,6cyclictest0-21swapper/311:35:463
254348199152,8cyclictest0-21swapper/311:30:513
254348199152,8cyclictest0-21swapper/310:00:503
254348199152,8cyclictest0-21swapper/310:00:503
254348199152,7cyclictest0-21swapper/312:20:503
254348199152,7cyclictest0-21swapper/311:00:483
254348199152,7cyclictest0-21swapper/308:15:453
254348199152,6cyclictest0-21swapper/309:00:493
254348199151,6cyclictest0-21swapper/307:10:483
254348199144,5cyclictest0-21swapper/311:20:453
254348199144,5cyclictest0-21swapper/311:20:453
254348199144,5cyclictest0-21swapper/311:10:393
254348199143,7cyclictest0-21swapper/310:45:503
254348199143,6cyclictest0-21swapper/311:45:493
254348199143,4cyclictest0-21swapper/307:35:553
254348199142,8cyclictest0-21swapper/312:30:423
254348199142,7cyclictest0-21swapper/312:10:513
254348199142,7cyclictest0-21swapper/310:55:473
254348199142,7cyclictest0-21swapper/310:30:513
254348199142,7cyclictest0-21swapper/309:20:463
254348199142,7cyclictest0-21swapper/309:20:463
254348199142,7cyclictest0-21swapper/309:15:463
254348199142,6cyclictest0-21swapper/310:50:363
254348199142,5cyclictest0-21swapper/310:20:343
254348199141,7cyclictest0-21swapper/310:05:523
254348199141,7cyclictest0-21swapper/310:05:523
254348199141,6cyclictest0-21swapper/308:30:443
254348199141,6cyclictest0-21swapper/308:20:443
254348199141,6cyclictest0-21swapper/307:25:493
254348199133,6cyclictest0-21swapper/309:25:353
254348199133,6cyclictest0-21swapper/309:25:353
254348199133,6cyclictest0-21swapper/309:10:453
254348199132,7cyclictest0-21swapper/311:15:483
254348199132,6cyclictest0-21swapper/309:55:523
254348199132,6cyclictest0-21swapper/309:30:363
254348199131,7cyclictest0-21swapper/312:35:523
254348199131,7cyclictest0-21swapper/310:10:363
254348199131,7cyclictest0-21swapper/308:40:013
254348199131,6cyclictest0-21swapper/311:55:463
254348199131,6cyclictest0-21swapper/309:05:483
254348199131,6cyclictest0-21swapper/308:55:473
254348199130,4cyclictest0-21swapper/308:10:513
254348199122,6cyclictest0-21swapper/312:05:423
254348199122,6cyclictest0-21swapper/312:00:493
254348199122,6cyclictest0-21swapper/312:00:493
254348199122,6cyclictest0-21swapper/311:05:353
254348199121,7cyclictest0-21swapper/312:25:413
254348199121,7cyclictest0-21swapper/310:35:333
254348199121,7cyclictest0-21swapper/309:35:513
254348199121,6cyclictest0-21swapper/310:40:463
254348199121,6cyclictest0-21swapper/310:40:463
254348199121,6cyclictest0-21swapper/307:50:533
254348199121,6cyclictest0-21swapper/307:50:533
254348199112,6cyclictest0-21swapper/311:50:363
254348199111,6cyclictest0-21swapper/307:55:483
254348199111,6cyclictest0-21swapper/307:55:483
254348199111,6cyclictest0-21swapper/307:45:473
254348199111,6cyclictest0-21swapper/307:45:473
254348199102,5cyclictest0-21swapper/308:40:393
254348199101,6cyclictest0-21swapper/307:15:413
254348199101,5cyclictest431ktimers/307:10:013
254348199101,5cyclictest0-21swapper/307:25:033
25434819995,3cyclictest0-21swapper/308:00:413
25434819995,3cyclictest0-21swapper/307:40:353
25434819991,6cyclictest0-21swapper/307:35:023
25434819991,5cyclictest0-21swapper/308:45:393
25434819991,5cyclictest0-21swapper/308:25:393
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional