You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-03-25 - 23:40
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7.osadl.org (updated Tue Mar 25, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2888697994522,6cyclictest3049676-21apt-get11:52:300
288869799421,6cyclictest2947550-21df09:40:510
2888697994020,3cyclictest3035023-21ssh11:35:560
2888697993924,5cyclictest0-21swapper/012:33:020
2888697993923,12cyclictest0-21swapper/011:07:560
2888697993922,2cyclictest0-21swapper/010:42:310
2888697993319,2cyclictest0-21swapper/012:00:560
2888697993018,4cyclictest3001318-21/usr/sbin/munin10:55:550
2888697992912,4cyclictest2957620-21ntp_states09:55:550
2888697992912,3cyclictest0-21swapper/010:01:040
288869999289,15cyclictest3040385-21ssh11:41:272
288869799280,1cyclictest2982322-21ssh10:30:510
288869799270,3cyclictest2922670-21idleruntime-cro08:52:270
288869799270,19cyclictest2986754-21unixbench_multi10:35:590
288869999262,17cyclictest2994908-21ssh10:47:282
288869799253,19cyclictest992-21lldpd12:27:420
2888697992516,2cyclictest0-21swapper/012:20:510
2888697992514,6cyclictest0-21swapper/010:47:280
288869799251,15cyclictest0-21swapper/012:06:260
2888699992416,7cyclictest3046414-21kworker/u8:1+events_unbound12:05:262
288869899246,7cyclictest11050irq/107-eth%d12:00:571
2888697992416,6cyclictest2971122-21gltestperf10:15:520
288869799241,19cyclictest0-21swapper/009:20:520
2888699992316,4cyclictest0-21swapper/212:22:402
288869899233,15cyclictest2929252-21diskmemload11:47:551
288869899231,3cyclictest171rcu_preempt11:40:541
288869799234,7cyclictest3068655-21/usr/sbin/munin12:12:270
288869799234,7cyclictest0-21swapper/010:20:520
288869799232,3cyclictest3017911-21/usr/sbin/munin11:15:520
288869999221,1cyclictest171rcu_preempt09:50:512
288869899223,12cyclictest111788-21systemd-journal07:28:521
2888698992216,3cyclictest0-21swapper/112:15:411
288869799222,13cyclictest1005-21mta-sts-daemon10:10:510
2888697992214,1cyclictest0-21swapper/009:35:590
288869799220,21cyclictest0-21swapper/011:20:580
288869999212,2cyclictest171rcu_preempt11:00:572
288869999212,16cyclictest3018785-21/usr/sbin/munin11:16:002
288869999212,12cyclictest11050irq/107-eth%d11:07:562
288869999211,4cyclictest0-21swapper/211:35:552
2888699992111,7cyclictest351ktimers/211:52:302
288869999211,16cyclictest3044651-21cron11:47:272
288869899215,14cyclictest619-21mta-sts-daemon11:00:571
288869899213,13cyclictest3095603-21/usr/sbin/munin12:40:491
288869899212,15cyclictest3018691-21ssh11:16:001
2888698992111,9cyclictest0-21swapper/111:20:531
288869799219,1cyclictest3027803-21grep11:27:280
2888697992112,2cyclictest0-21swapper/010:40:360
112699210,4rtkit-daemon1008-21mta-sts-daemon11:48:520
2888699992016,3cyclictest2940539-21fschecks_count09:30:462
288869899207,8cyclictest0-21swapper/109:50:511
288869899204,12cyclictest2973456-21ssh10:17:291
288869899203,13cyclictest2989951-21ntpq10:40:541
2888698992013,5cyclictest2962482-21idleruntime-cro10:02:271
288869799201,13cyclictest3011156-21unin-run11:07:280
288869799200,20cyclictest0-21swapper/010:59:280
112699200,1rtkit-daemon0-21swapper/207:07:372
288869999195,11cyclictest3088247-21sh12:33:022
2888699991912,4cyclictest0-21swapper/212:15:482
288869899193,12cyclictest11050irq/107-eth%d11:52:301
288869899192,14cyclictest3066510-21gltestperf12:10:471
288869799193,14cyclictest2909741-21apt-get08:12:290
2888697991912,2cyclictest2898809-21grep07:41:000
288869799190,19cyclictest0-21swapper/011:28:380
288869999184,2cyclictest3001937-21ssh10:55:532
288869999183,8cyclictest2949385-21munin-plugin-st09:42:272
288869999181,5cyclictest171rcu_preempt12:10:502
288869899187,6cyclictest3087872-21apt-get12:32:291
288869899186,11cyclictest1008-21mta-sts-daemon11:30:571
288869899183,10cyclictest2971035-21cut10:15:521
288869899181,8cyclictest3087480-21cat12:32:271
288869799187,4cyclictest2925130-21/usr/sbin/munin09:00:550
288869799184,11cyclictest2932185-21ssh09:15:590
288869799184,10cyclictest3043507-21ssh11:45:450
288869799182,6cyclictest2953589-21/usr/sbin/munin09:50:490
2888697991818,0cyclictest0-21swapper/010:05:200
288869799181,3cyclictest0-21swapper/012:40:520
288869799181,14cyclictest3037189-21sh11:38:300
2888697991811,1cyclictest0-21swapper/009:25:400
288869799180,12cyclictest0-21swapper/012:24:270
288869999173,2cyclictest3030940-21/usr/sbin/munin11:30:552
288869999172,9cyclictest3047160-21/usr/sbin/munin11:50:542
288869999172,9cyclictest111788-21systemd-journal10:02:412
288869899173,8cyclictest2902479-21awk07:52:281
288869899171,14cyclictest2982778-21sed10:30:541
2888698991711,3cyclictest2898940-21latency_hist07:42:271
288869899170,11cyclictest2957052-21cut09:55:511
288869799178,5cyclictest1008-21mta-sts-daemon07:12:270
112699170,0rtkit-daemon0-21swapper/210:43:162
288869999166,5cyclictest11050irq/107-eth%d10:15:552
288869999164,9cyclictest3058607-21ssh12:00:582
288869999164,8cyclictest3095209-21/usr/sbin/munin12:40:472
288869999163,11cyclictest3020395-21sshd11:17:302
288869999163,10cyclictest2960898-21/usr/sbin/munin10:00:532
2888699991612,3cyclictest2908076-21cut08:10:542
288869999161,14cyclictest2987122-21ssh10:36:362
288869899163,9cyclictest2944416-21/usr/sbin/munin09:35:531
288869899162,10cyclictest2904822-21cut08:00:541
288869799162,12cyclictest3072107-21rm12:15:550
2888697991616,0cyclictest0-21swapper/009:44:420
288869799160,7cyclictest2940370-21/usr/sbin/munin09:30:450
112699160,3rtkit-daemon1125-21rtkit-daemon10:09:231
112699160,3rtkit-daemon1125-21rtkit-daemon07:40:172
112699160,2rtkit-daemon1125-21rtkit-daemon08:35:441
288870099153,7cyclictest0-21swapper/311:44:413
288870099152,7cyclictest0-21swapper/310:10:513
288869999158,4cyclictest111788-21systemd-journal10:40:572
288869999156,7cyclictest2957296-21/usr/sbin/munin09:55:522
288869999155,5cyclictest3085334-21/usr/sbin/munin12:30:482
288869999153,8cyclictest2934807-21ssh09:20:492
288869999151,9cyclictest111788-21systemd-journal08:32:272
2888699991512,2cyclictest3027700-21idleruntime-cro11:27:282
2888699991512,2cyclictest2896853-21unixbench_multi07:36:012
288869999151,12cyclictest2902385-21apt-config07:52:282
2888699991510,4cyclictest2893528-21grep07:25:592
288869999150,3cyclictest0-21swapper/212:20:322
288869999150,12cyclictest2967243-21/usr/sbin/munin10:10:502
288869999150,11cyclictest2926500-21fschecks_count09:05:512
288869899157,3cyclictest2985750-21fschecks_time10:35:501
288869899155,8cyclictest3044653-21cron11:47:271
288869899153,8cyclictest3082304-21apt12:27:271
288869899152,9cyclictest3077129-21ssh12:20:561
288869899152,1cyclictest0-21swapper/111:04:351
2888698991511,3cyclictest2900590-21cron07:47:271
288869899151,12cyclictest3002480-21/usr/sbin/munin10:55:591
288869899150,3cyclictest2998044-21/usr/sbin/munin10:50:531
288869799157,2cyclictest2911540-21aten_r2power_po08:20:480
288869799153,8cyclictest2929156-21sed09:11:010
288869799153,10cyclictest2914803-21aten_r2power_fr08:30:510
288869799152,8cyclictest2902528-21gpgv07:52:280
112699150,1rtkit-daemon0-21swapper/007:21:000
288870099144,6cyclictest0-21swapper/311:50:523
288870099144,4cyclictest0-21swapper/311:42:273
288870099143,7cyclictest0-21swapper/310:47:283
288870099142,6cyclictest0-21swapper/312:10:473
288869999148,4cyclictest0-21swapper/208:45:482
288869999143,6cyclictest2889204-21grep07:15:502
288869999143,6cyclictest2889204-21grep07:15:502
288869999142,9cyclictest2907469-21grep08:06:002
288869999142,4cyclictest171rcu_preempt10:17:292
2888699991411,2cyclictest2905874-21idleruntime-cro08:02:272
288869899147,6cyclictest1125-21rtkit-daemon08:45:501
288869899145,7cyclictest2908038-21/usr/sbin/munin08:10:541
288869899144,8cyclictest0-21swapper/109:15:481
288869899142,10cyclictest2903990-21cat07:56:001
288869899141,9cyclictest2994621-21dpkg10:47:281
288869899141,9cyclictest2948631-21ssh09:41:001
2888698991411,2cyclictest1007-21mta-sts-daemon10:06:391
288869899140,2cyclictest2940261-21aten_r2power_en09:30:441
288869799140,9cyclictest2927240-21run-parts09:05:580
288869799140,2cyclictest0-21swapper/010:24:020
112699140,1rtkit-daemon0-21swapper/008:44:160
288870099133,6cyclictest0-21swapper/311:54:403
288870099133,4cyclictest0-21swapper/308:25:573
288870099132,6cyclictest0-21swapper/310:35:583
288870099132,6cyclictest0-21swapper/310:00:553
288869999134,6cyclictest2900169-21/usr/sbin/munin07:46:012
288869999133,9cyclictest2902809-21apt-get07:52:292
288869999133,8cyclictest2929074-21/usr/sbin/munin09:11:012
288869999133,5cyclictest2944348-21if_end009:35:532
288869999133,5cyclictest1006-21mta-sts-daemon09:46:082
288869999132,9cyclictest2981354-21/usr/sbin/munin10:30:432
288869999132,7cyclictest3008067-21sendmail-msp11:02:332
2888699991311,1cyclictest0-21swapper/209:15:492
2888699991310,2cyclictest2925901-21unixbench_singl09:01:022
288869899139,3cyclictest0-21swapper/110:27:271
288869899133,7cyclictest3025997-21ssh11:24:301
288869899132,8cyclictest3035295-21/usr/sbin/munin11:35:591
288869899132,7cyclictest2911130-21/usr/sbin/munin08:15:591
288869899132,7cyclictest2897143-21sed07:37:271
288869899131,9cyclictest2934751-21ssh09:20:491
2888698991310,2cyclictest3014473-21unixbench_multi11:10:551
2888698991310,2cyclictest1006-21mta-sts-daemon08:20:531
288869899130,11cyclictest2918081-21aten_r2power_fr08:40:501
288869799132,7cyclictest2923830-21/usr/sbin/munin08:55:580
288869799132,7cyclictest2889444-21/usr/sbin/munin07:15:510
288869799132,7cyclictest2889444-21/usr/sbin/munin07:15:510
288869799131,6cyclictest2906603-21sed08:05:520
288869799130,9cyclictest2897181-21latency_hist07:37:270
288870099125,3cyclictest0-21swapper/311:16:003
288870099124,4cyclictest0-21swapper/312:15:553
288870099123,3cyclictest0-21swapper/308:50:563
288870099122,5cyclictest0-21swapper/312:30:233
288870099122,5cyclictest0-21swapper/311:17:303
288870099122,5cyclictest0-21swapper/310:56:003
288870099122,5cyclictest0-21swapper/309:15:563
288869999127,4cyclictest2890851-21cut07:20:502
288869999121,9cyclictest2921822-21if_wlan008:50:522
288869999121,9cyclictest2911766-21tune2fs08:20:502
288869999121,9cyclictest2911022-21sed08:15:582
288869899128,3cyclictest0-21swapper/109:05:591
288869899125,4cyclictest2951788-21latency_hist09:47:271
288869899123,8cyclictest111788-21systemd-journal09:12:281
288869899122,8cyclictest2924907-21grep09:00:541
288869899121,8cyclictest2913642-21/usr/sbin/munin08:25:541
2888698991210,1cyclictest1005-21mta-sts-daemon08:50:571
288869799127,1cyclictest0-21swapper/008:25:500
288869799127,1cyclictest0-21swapper/007:25:520
288869799123,5cyclictest2918544-21/usr/sbin/munin08:40:530
288869799121,9cyclictest2902961-21aten_r2power_po07:55:500
288869799121,8cyclictest2894306-21/usr/sbin/munin07:30:540
288869799120,8cyclictest2900055-21ntpq07:45:590
288869799120,8cyclictest0-21swapper/008:10:530
288870099116,3cyclictest0-21swapper/312:00:563
288870099114,3cyclictest0-21swapper/311:33:063
288870099114,3cyclictest0-21swapper/309:50:503
288870099113,5cyclictest0-21swapper/309:20:563
288870099112,6cyclictest0-21swapper/309:40:533
288870099112,5cyclictest0-21swapper/312:33:023
288870099112,5cyclictest0-21swapper/311:05:473
288870099112,5cyclictest0-21swapper/309:55:513
288870099112,3cyclictest0-21swapper/308:35:563
288869999114,6cyclictest2893869-21apt-get07:28:512
288869999112,4cyclictest2917723-21/usr/sbin/munin08:35:592
288869999110,8cyclictest2918329-21fschecks_time08:40:522
288869999110,10cyclictest2923388-21gltestperf08:55:522
288869899113,5cyclictest2915059-21fschecks_time08:30:531
288869899111,7cyclictest2906436-21cut08:05:511
288869899111,7cyclictest2892729-21/usr/sbin/munin07:25:521
288869799116,4cyclictest0-21swapper/008:00:550
288870099104,4cyclictest0-21swapper/309:30:493
288870099103,4cyclictest0-21swapper/312:40:503
288870099103,4cyclictest0-21swapper/310:20:523
288870099103,3cyclictest0-21swapper/310:28:323
288870099103,3cyclictest0-21swapper/309:47:273
288870099102,6cyclictest0-21swapper/310:02:413
288870099102,4cyclictest0-21swapper/312:22:583
288870099102,4cyclictest0-21swapper/311:28:483
288870099102,4cyclictest0-21swapper/310:38:453
288870099102,3cyclictest0-21swapper/312:19:263
288870099101,6cyclictest0-21swapper/311:00:483
288870099101,6cyclictest0-21swapper/310:42:443
288870099101,5cyclictest0-21swapper/309:35:573
288870099101,5cyclictest0-21swapper/307:28:523
288869999108,1cyclictest0-21swapper/208:25:532
288869999107,2cyclictest11050irq/107-eth%d10:27:272
288869999102,5cyclictest2938868-21wc09:27:272
288869899107,2cyclictest0-21swapper/107:15:511
288869899107,2cyclictest0-21swapper/107:15:511
288869899102,5cyclictest2891857-21unixbench_multi07:20:581
288869899102,3cyclictest3060062-21sendmail-msp12:02:371
288869899101,6cyclictest2923298-21fschecks_time08:55:521
288869799104,3cyclictest2917329-21/usr/sbin/munin08:35:570
28887009995,3cyclictest0-21swapper/311:10:373
28887009992,4cyclictest0-21swapper/312:04:263
28887009992,4cyclictest0-21swapper/310:14:313
28887009991,4cyclictest0-21swapper/308:01:003
28887009991,4cyclictest0-21swapper/307:25:523
28886989991,2cyclictest1559842-1kworker/1:1H-kblockd09:22:391
28887009984,3cyclictest0-21swapper/307:35:583
28887009984,3cyclictest0-21swapper/307:15:563
28887009984,3cyclictest0-21swapper/307:15:563
28887009982,5cyclictest0-21swapper/307:52:283
28887009982,4cyclictest0-21swapper/308:40:583
28887009982,4cyclictest0-21swapper/308:30:543
28887009982,3cyclictest0-21swapper/308:20:493
28887009982,3cyclictest0-21swapper/307:45:543
28887009982,2cyclictest0-21swapper/311:22:303
28887009981,4cyclictest0-21swapper/308:45:523
28887009981,4cyclictest0-21swapper/308:12:293
28887009980,5cyclictest0-21swapper/309:00:533
28887009980,4cyclictest0-21swapper/310:27:183
28887009980,4cyclictest0-21swapper/308:55:573
28886989984,2cyclictest0-21swapper/107:12:271
28887009975,1cyclictest0-21swapper/309:05:553
28887009972,4cyclictest0-21swapper/308:10:523
28887009972,3cyclictest0-21swapper/308:05:583
28887009972,3cyclictest0-21swapper/307:40:523
28887009971,4cyclictest0-21swapper/309:27:273
28887009971,4cyclictest0-21swapper/309:10:513
28887009971,3cyclictest0-21swapper/307:20:483
28887009962,3cyclictest0-21swapper/307:52:293
28887009951,2cyclictest0-21swapper/307:12:273
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional