You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-03-24 - 00:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Sun Mar 23, 2025 12:42:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
761295995647,6cyclictest4164054-21cups-browsed07:11:581
76129599534,39cyclictest872641-21ssh11:17:031
761295995043,3cyclictest0-21swapper/111:01:591
761295995041,6cyclictest4164054-21cups-browsed07:16:561
761295994941,5cyclictest1-21systemd09:25:021
761295994842,3cyclictest0-21swapper/111:50:511
761294994839,5cyclictest0-21swapper/012:40:010
761296994738,6cyclictest799565-21systemd-run09:15:022
761294994736,8cyclictest902332-21ssh12:00:010
761295994620,5cyclictest0-21swapper/108:42:011
76129699453,35cyclictest848974-21ssh10:37:082
76129499454,34cyclictest797902-21ssh09:12:010
761295994436,3cyclictest0-21swapper/112:16:581
761295994335,5cyclictest838362-21idleruntime-cro10:20:011
761295994332,7cyclictest861026-21grep10:57:081
761296994234,5cyclictest775056-21sendmail08:00:012
761296994234,5cyclictest775056-21sendmail08:00:012
761296994228,10cyclictest872503-21ntp_kernel_pll_11:16:582
761295994234,5cyclictest1-21systemd12:40:021
761295994232,6cyclictest266-21systemd-journal09:20:021
761295994230,8cyclictest266-21systemd-journal07:10:021
761295994216,20cyclictest4164051-21cupsd11:12:011
76129499423,33cyclictest860497-21ssh10:56:570
761296994134,4cyclictest931366-21latency_hist12:40:032
76129699412,34cyclictest860576-21ssh10:57:002
761295994133,5cyclictest796051-21diskmemload09:37:081
761295994133,4cyclictest900878-21ssh11:57:111
761295994132,6cyclictest421-21avahi-daemon12:29:121
76129599412,32cyclictest881825-21ssh11:32:001
761294994134,3cyclictest0-21swapper/011:12:120
761296994032,5cyclictest867621-21munin-run11:10:012
761295994035,2cyclictest0-21swapper/111:40:011
761295994032,5cyclictest775089-21latency_hist08:00:021
761295994032,5cyclictest775089-21latency_hist08:00:021
761295994030,6cyclictest0-21swapper/108:47:031
761294994032,4cyclictest151ktimers/007:11:570
761296993928,8cyclictest767534-21cron07:35:012
761296993926,10cyclictest698-21cron11:40:012
761295993932,3cyclictest0-21swapper/110:25:011
761295993931,5cyclictest791544-21idleruntime-cro08:55:011
761295993930,4cyclictest959-21mta-sts-daemon10:12:271
761294993914,6cyclictest171rcu_preempt07:47:040
761296993832,3cyclictest0-21swapper/212:17:122
761296993830,5cyclictest826823-21latency_hist10:00:022
761296993830,5cyclictest807006-21sshd09:26:542
761295993831,3cyclictest0-21swapper/109:10:011
761295993830,6cyclictest829963-21cut10:05:011
761294993831,4cyclictest801016-21needreboot09:16:540
761294993829,6cyclictest880096-21latency_hist11:30:030
761294993829,6cyclictest880096-21latency_hist11:30:030
761294993825,9cyclictest855557-21sh10:50:020
761296993730,5cyclictest761293-21cyclictest10:42:022
761296993729,5cyclictest959-21mta-sts-daemon11:58:192
761296993729,5cyclictest815012-21sendmail09:40:012
761296993728,5cyclictest802781-21systemd-run09:20:022
761295993731,3cyclictest0-21swapper/111:10:011
761295993724,10cyclictest849338-21timerwakeupswit10:37:141
76129599372,32cyclictest825042-21ssh09:57:021
76129599371,7cyclictest0-21swapper/110:41:571
761294993730,4cyclictest788548-21cron08:45:010
761294993730,3cyclictest0-21swapper/008:48:500
761294993729,5cyclictest828601-21mailstats10:02:140
761294993727,7cyclictest916767-21ssh12:20:010
761294993726,6cyclictest151ktimers/007:31:530
761294993721,12cyclictest0-21swapper/012:22:020
76129699363,25cyclictest4164054-21cups-browsed09:52:002
761296993631,3cyclictest0-21swapper/212:10:012
761296993629,4cyclictest901784-21kworker/u8:10-rpciod12:15:012
761296993629,4cyclictest901784-21kworker/u8:10-rpciod12:15:012
761296993629,4cyclictest778066-21kworker/u8:2-rpciod08:48:492
761296993628,5cyclictest955-21mta-sts-daemon09:45:412
761296993628,5cyclictest422-21dbus-daemon11:25:022
76129599364,25cyclictest889663-21ssh11:41:581
76129599363,27cyclictest821865-21ssh09:51:571
76129599363,26cyclictest809944-21ssh09:31:581
761295993627,5cyclictest785534-21latency_hist08:35:011
761295993623,10cyclictest857531-21timerandwakeup10:52:041
761294993630,4cyclictest891753-21cron11:45:010
761294993617,14cyclictest12450irq/32-eth010:37:040
76129499361,29cyclictest872514-21sh11:16:580
761296993527,5cyclictest864082-21mailstats11:02:072
761296993527,5cyclictest806065-21sh09:25:012
761296993527,5cyclictest422-21dbus-daemon09:35:022
761296993525,6cyclictest818490-21latency_hist09:45:032
76129699351,26cyclictest896237-21ssh11:51:532
761295993530,2cyclictest0-21swapper/109:00:011
761295993528,3cyclictest0-21swapper/111:30:011
761295993528,3cyclictest0-21swapper/111:30:011
761295993522,9cyclictest0-21swapper/107:37:021
76129499353,27cyclictest836970-21ssh10:17:000
761294993529,4cyclictest826757-21cron10:00:010
761294993527,5cyclictest4164054-21cups-browsed12:31:010
761294993524,7cyclictest151ktimers/008:06:580
76129699345,23cyclictest889634-21ssh11:41:562
761296993428,3cyclictest0-21swapper/212:30:012
761296993426,5cyclictest920890-21latency_hist12:25:032
761296993425,6cyclictest957-21mta-sts-daemon10:22:132
76129699341,28cyclictest869268-21ssh11:11:592
761295993427,4cyclictest0-21swapper/111:25:011
761294993427,5cyclictest957-21mta-sts-daemon12:05:280
761294993427,4cyclictest887864-21systemd-run11:40:010
761294993426,5cyclictest910729-21ntp_kernel_pll_12:11:530
761294993426,5cyclictest910729-21ntp_kernel_pll_12:11:530
761294993426,5cyclictest875734-21sendmail_mailqu11:22:050
761294993421,10cyclictest817168-21unixbench_singl09:42:080
76129699333,25cyclictest956-21mta-sts-daemon08:52:072
761296993326,4cyclictest959-21mta-sts-daemon10:10:242
761296993326,3cyclictest0-21swapper/211:32:112
76129599337,4cyclictest0-21swapper/108:07:011
761295993327,3cyclictest0-21swapper/107:30:011
761295993326,4cyclictest912876-21idleruntime-cro12:15:001
761295993326,4cyclictest912876-21idleruntime-cro12:15:001
761295993325,5cyclictest787046-21idleruntime-cro08:40:011
761295993323,6cyclictest4164054-21cups-browsed10:31:061
761295993322,7cyclictest421-21avahi-daemon11:49:341
76129599332,26cyclictest796051-21diskmemload12:22:051
76129599332,26cyclictest421-21avahi-daemon09:41:581
76129599332,26cyclictest421-21avahi-daemon07:51:581
76129599332,26cyclictest421-21avahi-daemon07:51:581
76129599332,22cyclictest904339-21ssh12:02:031
761295993317,13cyclictest271ktimers/107:47:041
76129499336,20cyclictest904233-21ssh12:02:000
761294993326,5cyclictest0-21swapper/009:40:020
761294993326,3cyclictest0-21swapper/012:30:020
761294993324,7cyclictest832139-21munin-plugin-st10:10:020
761294993320,10cyclictest0-21swapper/008:20:010
761296993226,3cyclictest0-21swapper/208:45:012
761296993225,3cyclictest0-21swapper/210:05:012
761296993224,5cyclictest781045-21sendmail08:20:012
761296993224,4cyclictest0-21swapper/210:16:512
761295993225,4cyclictest765793-21munin-run07:25:011
761295993224,5cyclictest780077-21fschecks_time08:16:551
76129499324,18cyclictest840117-21ssh10:22:020
761294993227,3cyclictest852093-21ntp_kernel_err10:41:570
761294993223,6cyclictest844796-21aten_r2power_en10:31:470
761296993126,2cyclictest0-21swapper/207:15:012
761296993124,3cyclictest0-21swapper/210:55:022
761296993123,4cyclictest0-21swapper/212:02:052
761296993123,4cyclictest0-21swapper/211:30:022
761296993123,4cyclictest0-21swapper/211:30:022
761296993121,6cyclictest894489-21unin-run11:50:012
761295993122,6cyclictest453-21systemd-logind12:35:021
76129499313,22cyclictest833707-21ssh10:12:220
761294993125,4cyclictest770484-21unixbench_singl07:42:110
761294993125,3cyclictest0-21swapper/007:20:010
761294993120,8cyclictest0-21swapper/011:05:010
761296993025,3cyclictest0-21swapper/208:05:012
761296993023,4cyclictest957-21mta-sts-daemon08:32:032
761296993023,4cyclictest769590-21fschecks_time07:41:532
761296993022,5cyclictest266-21systemd-journal10:50:012
76129699302,22cyclictest847150-21ssh10:34:542
761295993022,5cyclictest908956-21grep12:10:011
761295993021,7cyclictest766754-21latency07:31:501
761294993024,4cyclictest867622-21munin-plugin-st11:10:010
761294993023,4cyclictest811625-21munin-plugin-st09:35:010
761294993022,5cyclictest796053-21munin-plugin-st09:10:010
761294993022,5cyclictest422-21dbus-daemon07:55:010
761294993022,5cyclictest422-21dbus-daemon07:55:010
76129499302,23cyclictest858079-21sshd10:52:550
761294993021,6cyclictest4164054-21cups-browsed11:53:580
761294993020,4cyclictest959-21mta-sts-daemon09:23:330
76129699299,13cyclictest959-21mta-sts-daemon12:31:582
761296992923,3cyclictest955-21mta-sts-daemon07:36:572
761296992922,4cyclictest794550-21sh09:05:012
76129699292,17cyclictest437-21mta-sts-daemon08:22:042
761296992921,5cyclictest955-21mta-sts-daemon07:17:002
761295992922,3cyclictest0-21swapper/110:30:031
761295992921,5cyclictest855526-21latency_hist10:50:021
761295992921,5cyclictest775295-21/usr/sbin/munin08:01:591
761294992923,4cyclictest881392-21fschecks_count11:31:520
761294992921,6cyclictest784158-21latency_hist08:30:020
761294992913,13cyclictest0-21swapper/009:51:510
761296992821,4cyclictest787244-21latency_hist08:40:032
761296992820,5cyclictest956-21mta-sts-daemon10:29:162
761296992819,4cyclictest0-21swapper/208:56:332
761295992823,2cyclictest0-21swapper/108:27:121
761295992821,4cyclictest956-21mta-sts-daemon09:45:421
761294992821,3cyclictest0-21swapper/009:48:590
761294992821,3cyclictest0-21swapper/007:56:590
761294992821,3cyclictest0-21swapper/007:56:590
761296992720,5cyclictest0-21swapper/208:15:032
76129699271,20cyclictest795410-21sshd09:06:542
761295992720,4cyclictest779533-21cron08:15:011
761295992720,3cyclictest0-21swapper/109:11:381
761294992723,2cyclictest0-21swapper/007:25:020
761294992721,3cyclictest0-21swapper/008:40:010
761294992718,7cyclictest779546-21cut08:15:020
761294992714,11cyclictest793518-21cpu09:01:500
761296992620,3cyclictest0-21swapper/208:27:122
761294992614,9cyclictest782451-21unixbench_multi08:22:160
761296992519,3cyclictest0-21swapper/207:10:022
761296992517,5cyclictest773542-21idleruntime-cro07:55:012
761296992517,5cyclictest773542-21idleruntime-cro07:55:012
761296992517,5cyclictest771619-21sendmail_mailqu07:47:062
761296992517,4cyclictest0-21swapper/210:06:262
76129599256,3cyclictest171rcu_preempt08:22:111
761295992517,6cyclictest957-21mta-sts-daemon09:25:441
76129499258,14cyclictest422-21dbus-daemon08:55:010
761294992510,11cyclictest4164054-21cups-browsed10:29:490
761296992418,3cyclictest0-21swapper/207:30:012
761295992417,4cyclictest957-21mta-sts-daemon10:07:371
761295992417,4cyclictest957-21mta-sts-daemon07:41:111
761295992417,4cyclictest794422-21unixbench-2d09:02:121
76129499249,11cyclictest892962-21sh11:47:000
761294992419,3cyclictest956-21mta-sts-daemon08:57:110
761294992418,4cyclictest768941-21unixbench_multi07:37:090
761294992417,4cyclictest785452-21tail08:32:070
761294992317,3cyclictest0-21swapper/007:30:020
761294992316,5cyclictest0-21swapper/008:02:090
76129499220,10cyclictest808371-21date09:30:030
76129799213,9cyclictest0-21swapper/311:37:043
761296992115,3cyclictest0-21swapper/207:25:012
76129699201,14cyclictest777059-21diskstats08:06:522
76129799192,8cyclictest0-21swapper/311:41:563
76129799183,8cyclictest0-21swapper/311:51:543
76129799181,9cyclictest0-21swapper/309:51:553
76129799173,9cyclictest0-21swapper/312:11:553
76129799173,9cyclictest0-21swapper/312:11:553
76129799173,9cyclictest0-21swapper/309:41:543
76129799173,8cyclictest0-21swapper/310:31:523
76129799173,8cyclictest0-21swapper/309:12:023
76129799173,6cyclictest0-21swapper/308:42:013
76129799172,9cyclictest0-21swapper/310:22:043
76129799172,8cyclictest0-21swapper/311:31:593
76129799172,8cyclictest0-21swapper/311:16:593
76129799172,10cyclictest0-21swapper/312:02:043
76129499179,6cyclictest0-21swapper/007:10:030
76129799163,8cyclictest0-21swapper/310:56:553
76129799163,8cyclictest0-21swapper/310:12:233
76129799162,9cyclictest0-21swapper/307:13:553
76129799162,8cyclictest0-21swapper/310:01:493
76129799162,8cyclictest0-21swapper/309:21:573
76129799162,7cyclictest0-21swapper/307:22:163
76129799161,9cyclictest0-21swapper/312:31:553
76129799161,9cyclictest0-21swapper/312:16:583
76129799161,9cyclictest0-21swapper/309:16:563
76129799161,8cyclictest0-21swapper/308:27:043
76129799161,6cyclictest0-21swapper/308:57:003
76129799154,4cyclictest0-21swapper/308:52:053
76129799153,7cyclictest0-21swapper/311:01:593
76129799153,7cyclictest0-21swapper/309:31:553
76129799152,8cyclictest0-21swapper/310:51:523
76129799152,8cyclictest0-21swapper/310:37:023
76129799152,7cyclictest0-21swapper/312:36:563
76129799152,7cyclictest0-21swapper/310:16:593
76129799152,6cyclictest0-21swapper/307:46:553
76129799151,8cyclictest0-21swapper/310:41:583
76129799151,8cyclictest0-21swapper/309:37:003
76129799151,7cyclictest0-21swapper/309:06:563
76129799143,7cyclictest0-21swapper/312:22:013
76129799143,6cyclictest0-21swapper/311:21:573
76129799142,7cyclictest0-21swapper/311:45:553
76129799142,7cyclictest0-21swapper/308:48:483
76129799142,7cyclictest0-21swapper/307:56:583
76129799142,7cyclictest0-21swapper/307:56:583
76129799142,6cyclictest0-21swapper/310:06:513
76129799142,5cyclictest0-21swapper/307:31:543
76129799141,7cyclictest0-21swapper/312:00:033
76129799141,7cyclictest0-21swapper/309:02:063
76129799133,6cyclictest0-21swapper/311:11:483
76129799132,7cyclictest0-21swapper/307:16:543
76129799132,6cyclictest0-21swapper/308:22:043
76129799132,6cyclictest0-21swapper/307:37:003
76129799132,5cyclictest0-21swapper/308:12:023
76129799131,7cyclictest0-21swapper/307:51:593
76129799131,7cyclictest0-21swapper/307:51:593
76129799131,6cyclictest0-21swapper/308:31:553
76129799122,7cyclictest0-21swapper/309:56:493
76129799122,6cyclictest0-21swapper/312:30:023
76129799122,6cyclictest0-21swapper/311:29:543
76129799122,6cyclictest0-21swapper/311:29:543
76129799121,7cyclictest0-21swapper/311:10:033
76129799121,7cyclictest0-21swapper/310:26:323
76129799121,7cyclictest0-21swapper/308:16:263
76129799121,6cyclictest0-21swapper/309:26:543
76129799121,6cyclictest0-21swapper/308:37:073
76129799111,7cyclictest0-21swapper/310:49:553
76129799111,6cyclictest0-21swapper/312:10:023
76129799111,6cyclictest0-21swapper/308:06:523
76129799111,6cyclictest0-21swapper/308:01:573
76129799101,6cyclictest0-21swapper/307:41:593
76129799101,5cyclictest0-21swapper/309:50:013
7612979991,5cyclictest0-21swapper/307:30:013
7612979980,4cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional