You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-25 - 19:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sat Jul 25, 2026 12:45:20)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2876316999127,46cyclictest2946906-21ssh11:26:070
2876316998418,6cyclictest16850irq/134-800000010:26:500
2876316998322,15cyclictest2920974-21ssh10:31:060
2876327998272,7cyclictest37-21ksoftirqd/309:40:013
2876316998226,14cyclictest2933665-21ssh10:58:170
2876321998023,19cyclictest2938426-21ssh11:08:181
2876316998025,13cyclictest2900362-21rm09:47:310
2876316998023,17cyclictest2923482-21ssh10:36:260
2876321997923,16cyclictest2898022-21ssh09:42:341
2876321997923,16cyclictest2898022-21ssh09:42:341
2876321997820,19cyclictest2878798-21cat07:55:021
2876316997828,33cyclictest2909729-21ssh10:07:220
2876316997827,47cyclictest2935914-21ssh11:02:520
2876316997827,46cyclictest2970036-21ssh12:15:390
2876321997723,20cyclictest2891442-21ssh09:28:441
287631699772,6cyclictest2908542-21munin-run10:05:010
2876316997725,47cyclictest16950irq/224-800000012:11:500
2876321997631,9cyclictest2892738-21ssh09:31:081
2876316997629,43cyclictest2914030-21ssh10:16:170
2876316997628,44cyclictest2925801-21ssh10:41:150
2876316997619,40cyclictest16850irq/134-800000012:37:030
2876321997523,15cyclictest2910273-21ssh10:08:321
2876321997521,20cyclictest2902005-21ssh09:50:511
2876316997529,29cyclictest2885186-21ssh09:15:070
287631699752,69cyclictest2974372-21unin-run12:25:000
2876316997522,5cyclictest364-21snmpd11:32:580
2876316997517,12cyclictest2940482-21ssh11:12:370
287631699751,70cyclictest0-21swapper/010:45:030
287631699751,5cyclictest2932039-21latency_hist10:55:000
2876321997426,8cyclictest2936861-21munin-run11:05:021
2876321997421,15cyclictest2945788-21ssh11:24:051
2876321997419,21cyclictest2948978-21ssh11:30:321
2876321997419,16cyclictest2920585-21ssh10:30:081
2876316997433,24cyclictest16850irq/134-800000011:59:410
2876316997427,43cyclictest2893387-21ssh09:32:450
2876316997425,44cyclictest16850irq/134-800000010:00:000
287631699741,69cyclictest2887962-21ssh09:21:060
287632599731,51cyclictest291rcuc/210:05:022
2876321997326,13cyclictest2915531-21ssh10:19:451
2876321997321,16cyclictest2965467-21ssh12:05:511
2876321997320,14cyclictest2959621-21ssh11:54:021
2876316997333,37cyclictest16950irq/224-800000009:00:000
2876316997330,38cyclictest3467602-21systemd-journal11:35:570
2876316997326,2cyclictest0-21swapper/009:51:370
2876316997324,3cyclictest0-21swapper/011:15:480
2876321997221,14cyclictest2899291-21ssh09:45:051
2876321997221,14cyclictest2883410-21ssh09:11:311
2876321997220,15cyclictest2948394-21ssh11:29:491
2876321997219,21cyclictest2970025-21ssh12:15:371
2876321997219,17cyclictest2979788-21ssh12:36:181
2876321997218,16cyclictest2952035-21sh11:37:231
2876321997218,15cyclictest2894713-21ssh09:35:221
2876316997229,38cyclictest16950irq/224-800000009:44:390
2876316997229,38cyclictest16950irq/224-800000009:44:390
2876316997225,14cyclictest2966406-21ssh12:08:130
2876316997225,14cyclictest2913187-21ssh10:14:500
2876316997219,12cyclictest2963881-21ssh12:02:410
287631699721,26cyclictest0-21swapper/011:20:470
2876321997122,17cyclictest2931946-21ssh10:54:501
2876321997121,14cyclictest2925745-21ssh10:41:061
2876321997116,17cyclictest2941022-21ssh11:13:551
2876316997124,14cyclictest2978439-21ssh12:33:340
2876316997120,11cyclictest2917153-21ssh10:23:010
2876316997117,12cyclictest2955251-21ssh11:44:490
2876321997022,16cyclictest2967879-21ssh12:11:081
2876321997021,15cyclictest2919736-21sh10:28:391
2876321997020,14cyclictest2954770-21ssh11:43:391
2876321997019,15cyclictest2932369-21ssh10:55:141
2876321997019,13cyclictest2906842-21ssh10:01:061
2876321997016,16cyclictest2974080-21ssh12:24:201
2876316997025,5cyclictest0-21swapper/011:49:420
2876316997025,5cyclictest0-21swapper/011:49:420
2876316997025,12cyclictest2958829-21ssh11:52:060
2876321996921,13cyclictest2961435-21sh11:57:431
2876321996917,16cyclictest2943357-21ssh11:18:561
2876321996915,16cyclictest2916017-21ssh10:20:241
2876316996920,15cyclictest2894847-21ssh09:35:400
2876321996817,15cyclictest2975407-21ssh12:26:571
2876321996715,20cyclictest2913096-21ssh10:14:391
2876316996728,32cyclictest2879937-21sendmail08:20:000
2876316996728,32cyclictest2879937-21sendmail08:20:000
287631699671,11cyclictest2939376-21sed11:10:020
2876321996620,15cyclictest2885602-21ssh09:16:121
287631699662,6cyclictest2884922-21munin-run09:15:000
2876321996525,14cyclictest364-21snmpd09:07:391
2876321996524,14cyclictest2956749-21sh11:47:541
2876321996524,14cyclictest2956749-21sh11:47:541
2876316996523,4cyclictest0-21swapper/009:28:430
2876321996417,17cyclictest2927873-21ssh10:45:401
2876316996414,8cyclictest309-21in:imuxsock08:35:020
2876321996322,10cyclictest0-21swapper/109:56:141
2876321996317,14cyclictest2924481-21ssh10:38:501
2876321996316,14cyclictest2977299-21ssh12:30:481
2876316996317,11cyclictest2976347-21ssh12:29:080
2876325996212,7cyclictest131rcu_preempt09:30:002
2876321996118,14cyclictest2889043-21ssh09:23:331
2876327996046,10cyclictest3467602-21systemd-journal11:40:003
2876325996038,17cyclictest275-21dbus-daemon12:05:012
2876321996024,14cyclictest364-21snmpd12:03:311
2876316995928,28cyclictest0-21swapper/007:20:000
2876325995838,14cyclictest2906169-21tr10:00:002
287631699583,24cyclictest2876318-21latency_hist07:10:020
287631699583,24cyclictest2876318-21latency_hist07:10:020
2876325995748,5cyclictest2875900-21kworker/2:009:48:212
2876325995747,6cyclictest2882564-21kworker/2:310:31:472
2876325995740,13cyclictest275-21dbus-daemon08:40:002
2876316995727,11cyclictest12-21ksoftirqd/008:55:010
2876325995647,6cyclictest2957744-21kworker/2:112:10:402
2876325995646,6cyclictest2957744-21kworker/2:111:55:072
2876325995644,8cyclictest2887288-21systemctl09:20:002
2876321995641,11cyclictest2880480-21munin-run08:30:011
2876321995639,13cyclictest2878932-21munin-run08:00:011
2876316995624,2cyclictest0-21swapper/008:38:280
2876316995623,12cyclictest12-21ksoftirqd/007:55:000
2876325995547,5cyclictest2957744-21kworker/2:112:34:092
2876325995547,5cyclictest2882564-21kworker/2:309:25:042
2876325995546,5cyclictest2882564-21kworker/2:310:38:062
2876325995446,5cyclictest2957744-21kworker/2:112:07:442
2876325995445,6cyclictest2957744-21kworker/2:112:37:152
2876325995445,5cyclictest2882564-21kworker/2:310:17:442
2876321995440,11cyclictest364-21snmpd07:47:011
2876321995440,11cyclictest2879388-21sshd08:05:281
2876316995422,5cyclictest0-21swapper/008:45:000
287631699541,4cyclictest0-21swapper/007:47:280
287631699541,26cyclictest0-21swapper/007:41:240
2876325995345,5cyclictest2957744-21kworker/2:111:52:492
2876325995345,5cyclictest2908578-21kworker/2:211:34:522
2876325995345,5cyclictest2908578-21kworker/2:211:28:052
2876325995344,6cyclictest2875900-21kworker/2:009:10:342
2876325995344,6cyclictest2875900-21kworker/2:007:50:022
2876325995344,5cyclictest2908578-21kworker/2:211:21:292
2876325995343,5cyclictest2882564-21kworker/2:310:10:112
2876321995340,10cyclictest3467602-21systemd-journal07:42:411
2876321995340,10cyclictest2879687-21munin-run08:15:011
2876321995339,11cyclictest1-21systemd08:45:011
2876321995339,10cyclictest364-21snmpd07:32:381
2876321995335,15cyclictest364-21snmpd07:37:391
2876321995324,24cyclictest2882056-21wc09:00:011
2876316995320,7cyclictest12-21ksoftirqd/007:59:290
2876327995235,13cyclictest1-21systemd08:55:003
2876327995234,14cyclictest275-21dbus-daemon08:45:013
2876325995245,4cyclictest2908578-21kworker/2:211:39:232
2876325995244,5cyclictest2908578-21kworker/2:211:46:122
2876325995244,5cyclictest2908578-21kworker/2:211:46:112
2876325995244,5cyclictest2882564-21kworker/2:310:20:162
2876325995244,5cyclictest2882564-21kworker/2:310:09:572
2876325995244,5cyclictest2875900-21kworker/2:008:25:022
2876325995243,5cyclictest2957744-21kworker/2:112:25:052
2876325995243,5cyclictest2875900-21kworker/2:009:40:322
2876325995243,5cyclictest2875900-21kworker/2:009:40:322
2876325995235,12cyclictest2878896-21run-parts08:00:002
2876321995240,9cyclictest2879153-21sshd08:04:281
2876321995239,9cyclictest2882288-21munin-run09:05:001
2876321995231,14cyclictest275-21dbus-daemon07:20:011
2876321995226,20cyclictest2876686-21grep07:15:021
2876316995215,10cyclictest12-21ksoftirqd/008:08:070
287632799512,22cyclictest361rcuc/307:40:023
2876325995144,4cyclictest2908578-21kworker/2:211:13:532
2876325995143,5cyclictest2908578-21kworker/2:211:05:252
2876325995143,5cyclictest2908578-21kworker/2:211:03:232
2876325995143,5cyclictest2875900-21kworker/2:009:52:422
2876325995143,5cyclictest2875900-21kworker/2:009:05:022
2876325995142,5cyclictest2957744-21kworker/2:112:16:352
2876325995141,7cyclictest2908578-21kworker/2:211:15:042
2876321995138,9cyclictest364-21snmpd08:31:231
2876321995133,14cyclictest364-21snmpd08:22:051
2876321995126,19cyclictest2881164-21latency_hist08:40:021
2876316995115,10cyclictest12-21ksoftirqd/007:30:020
2876325995043,5cyclictest2875900-21kworker/2:007:25:022
2876325995042,5cyclictest2882564-21kworker/2:310:46:562
2876325995042,5cyclictest2882564-21kworker/2:310:26:152
2876325995042,5cyclictest2875900-21kworker/2:009:35:032
2876325995041,5cyclictest2882564-21kworker/2:310:53:302
2876325995041,5cyclictest2882564-21kworker/2:310:44:302
2876321995038,9cyclictest3467602-21systemd-journal08:53:551
2876321995036,10cyclictest2881501-21munin-run08:50:001
287631699501,26cyclictest0-21swapper/008:03:280
2876325994940,6cyclictest2875900-21kworker/2:008:42:522
2876325994940,5cyclictest2908578-21kworker/2:211:40:082
2876325994940,5cyclictest2882564-21kworker/2:310:55:282
2876321994929,15cyclictest364-21snmpd07:24:151
287631699492,23cyclictest0-21swapper/009:02:290
287631699491,5cyclictest1-21systemd07:34:280
2876325994840,5cyclictest2957744-21kworker/2:112:20:452
2876325994840,5cyclictest2875900-21kworker/2:008:12:342
2876325994840,5cyclictest2875900-21kworker/2:007:55:002
2876325994839,6cyclictest2875900-21kworker/2:007:35:002
2876321994834,10cyclictest3467602-21systemd-journal08:18:271
2876321994834,10cyclictest3467602-21systemd-journal08:18:261
2876316994831,11cyclictest2882736-21ssh09:10:020
2876327994732,11cyclictest275-21dbus-daemon11:35:013
2876327994731,12cyclictest1-21systemd10:20:013
2876325994739,5cyclictest2875900-21kworker/2:008:19:282
2876325994739,5cyclictest2875900-21kworker/2:008:19:282
2876325994739,5cyclictest2875900-21kworker/2:008:01:282
2876325994638,5cyclictest2875900-21kworker/2:009:33:232
2876325994638,5cyclictest2875900-21kworker/2:008:35:032
2876321994632,10cyclictest364-21snmpd07:25:331
2876327994534,7cyclictest1-21systemd10:35:013
2876327994528,12cyclictest3467602-21systemd-journal10:00:023
2876325994538,4cyclictest2875900-21kworker/2:008:52:452
2876325994536,6cyclictest2875900-21kworker/2:007:28:282
2876325994530,10cyclictest1-21systemd07:40:002
2876325994529,8cyclictest309-21in:imuxsock08:50:012
2876327994428,12cyclictest1-21systemd07:35:013
2876327994428,10cyclictest2878827-21t-helper07:59:293
287632799441,3cyclictest131rcu_preempt12:40:023
2876325994437,5cyclictest2875900-21kworker/2:008:29:152
2876325994436,5cyclictest2875900-21kworker/2:007:12:202
2876316994423,4cyclictest0-21swapper/007:20:420
2876327994333,7cyclictest2918757-21sshd10:26:283
2876327994332,8cyclictest37-21ksoftirqd/309:47:023
2876327994331,6cyclictest309-21in:imuxsock11:54:593
2876327994329,11cyclictest2955535-21sed11:45:033
2876327994329,10cyclictest1-21systemd07:50:023
287632799432,3cyclictest131rcu_preempt11:15:553
287632799431,3cyclictest131rcu_preempt10:20:103
2876327994231,7cyclictest37-21ksoftirqd/312:23:553
2876327994221,15cyclictest275-21dbus-daemon07:25:013
2876325994234,5cyclictest2875900-21kworker/2:008:05:202
2876316994232,7cyclictest2880193-21sshd08:22:280
287631699421,26cyclictest0-21swapper/007:38:160
2876327994129,7cyclictest2925026-21sendmail10:40:003
2876327994127,10cyclictest37-21ksoftirqd/311:01:573
2876327994126,10cyclictest1-21systemd08:05:013
287632799412,3cyclictest131rcu_preempt10:58:073
2876325994133,5cyclictest2875900-21kworker/2:009:06:182
2876325994132,6cyclictest2875900-21kworker/2:007:20:002
2876325994128,9cyclictest1-21systemd09:00:012
2876316994131,6cyclictest19350irq/34-mmc107:13:530
2876316994130,8cyclictest12-21ksoftirqd/008:15:010
287631699411,36cyclictest2880473-21systemd-run08:30:000
287632799409,20cyclictest0-21swapper/310:10:023
2876327994029,9cyclictest37-21ksoftirqd/312:02:093
2876327994026,11cyclictest275-21dbus-daemon11:25:003
287632799402,4cyclictest131rcu_preempt10:50:013
2876327994024,10cyclictest2882537-21cron09:10:003
2876327994022,15cyclictest1-21systemd09:20:003
2876327993930,6cyclictest37-21ksoftirqd/309:02:293
2876327993929,8cyclictest37-21ksoftirqd/310:13:013
2876327993929,6cyclictest2892122-21latency_hist09:30:023
2876327993927,7cyclictest2957690-21munin-plugin-st11:50:003
2876327993927,7cyclictest2957690-21munin-plugin-st11:50:003
2876327993927,7cyclictest2932194-21cat10:55:023
2876327993926,10cyclictest37-21ksoftirqd/309:40:283
2876327993926,10cyclictest37-21ksoftirqd/309:40:273
2876327993926,10cyclictest3467602-21systemd-journal08:56:283
2876327993925,10cyclictest37-21ksoftirqd/312:15:413
2876325993922,13cyclictest3467602-21systemd-journal07:40:022
2876321993923,12cyclictest2876386-21sed07:10:021
2876321993923,12cyclictest2876386-21sed07:10:021
287632799386,5cyclictest131rcu_preempt12:09:303
287632799383,15cyclictest3467602-21systemd-journal07:15:003
2876327993830,5cyclictest2884922-21/usr/sbin/munin09:15:013
2876327993829,6cyclictest2948596-21ssh11:30:033
2876327993827,8cyclictest37-21ksoftirqd/312:28:263
2876327993827,7cyclictest2879962-21sendmail-msp08:20:003
2876327993827,7cyclictest2879962-21sendmail-msp08:20:003
2876327993725,9cyclictest37-21ksoftirqd/311:14:203
2876327993722,11cyclictest3467602-21systemd-journal07:20:003
2876327993722,11cyclictest310-21in:imklog12:10:373
2876327993716,15cyclictest275-21dbus-daemon08:35:023
287632799363,3cyclictest131rcu_preempt11:59:323
2876327993629,4cyclictest2904905-21sh09:57:073
2876327993628,5cyclictest2887908-21sh09:20:593
2876327993628,5cyclictest2881653-21latency_hist08:50:023
2876327993624,9cyclictest2903908-21sed09:55:033
2876327993622,11cyclictest275-21dbus-daemon08:15:003
287632799358,23cyclictest1-21systemd09:35:013
2876327993526,7cyclictest37-21ksoftirqd/310:41:383
2876327993525,8cyclictest37-21ksoftirqd/312:32:513
2876327993521,11cyclictest37-21ksoftirqd/311:05:563
2876327993521,10cyclictest3467602-21systemd-journal08:20:043
2876327993510,7cyclictest131rcu_preempt07:45:003
287631699351,4cyclictest2881498-21modprobe08:50:000
2876327993421,9cyclictest2880646-21grep08:30:023
2876327993419,12cyclictest275-21dbus-daemon07:30:003
2876327993224,5cyclictest2876430-21latency_hist07:10:023
2876327993224,5cyclictest2876430-21latency_hist07:10:023
2876327993221,7cyclictest2880957-21cron08:39:593
2876327993120,8cyclictest37-21ksoftirqd/307:50:023
287632799301,3cyclictest131rcu_preempt08:10:013
2876325992517,5cyclictest2876250-21latency_hist07:10:022
2876325992517,5cyclictest2876250-21latency_hist07:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional