Creating a "Mathis" R library?

Just wanted to throw this out there as a possible project down the road. It is technically beyond me at the moment.

This project would entail creating an "R" package for Mathis's calculations.

One package in R that is really interesting is the circular package and how it could be used to set-up a Mathis' style molecular bonding system.

https://cran.r-project.org/web/packages/circular/circular.pdf

Circular Statistics Definitions:
-------
http://www.ncss.com/wp-content/themes/ncss/pdf/Procedures/NCSS/Circular_Data_Analysis.pdf
http://webspace.ship.edu/pgmarr/Geo441/Lectures/Lec%2016%20-%20Directional%20Statistics.pdf
https://en.wikipedia.org/wiki/Directional_statistics

Cr6, I looked at the references. Is R a statistical application or a math based library? I don't know enough to give you change for a dollar.

LongtimeAirman wrote:Cr6, I looked at the references. Is R a statistical application or a math based library? I don't know enough to give you change for a dollar.

Yes, it is Stats program like SAS/SPSS crossed with a Math program like MatLab. It is freeware and has a mix of stats libraries for running on various databases/mongo/unstructured data/json/etc. The cool thing is that it allows local development that can then be ported to bigger database/big data type systems. A lot of people do machine learning with it.

Also, R apps can be hosted on web servers like this:

http://opencpu.ocpu.io/stocks/www/

https://www.opencpu.org/demo.html

This is kind of where it can go. DeepLearning may provide a few tricks to hack Mathis and create "converter" tools to map to typical Chem notation.

http://www.slideshare.net/0xdata/h2-o-deeplearningarnocandel031115?next_slideshow=1

Cr6, Deep Learning. We're nowhere near it. Are we?

I can imagine nuclear charge flow in some aggregate form, crystalline streams that somehow maintain coherence, perhaps due to the presence of a wide range of energetic particles, including larger and slower ones like electrons, also recycling charge. Unconsciously, I think of atomic matter as charge field plumbing; I've often used plumbing analogies to describe electrical stuff to people. But that must be wrong. As Nevyn recently pointed out, it involves the power of non-linearity.

In the AC Current paper, Miles said we actually have two-way spun charge flow. Well, since he said it, there are algorithms waiting to be written to do that. That's one example.

Do you believe that we can assemble a library of MM algorithms?
.

LongtimeAirman wrote:Cr6, Deep Learning. We're nowhere near it. Are we?

I can imagine nuclear charge flow in some aggregate form, crystalline streams that somehow maintain coherence, perhaps due to the presence of a wide range of energetic particles, including larger and slower ones like electrons, also recycling charge. Unconsciously, I think of atomic matter as charge field plumbing; I've often used plumbing analogies to describe electrical stuff to people. But that must be wrong. As Nevyn recently pointed out, it involves the power of non-linearity.

In the AC Current paper, Miles said we actually have two-way spun charge flow. Well, since he said it, there are algorithms waiting to be written to do that. That's one example.

Do you believe that we can assemble a library of MM algorithms?
.

That's a great question LTAM and your comment on the charge field is a good one IMHO. I do think that getting even a primitive "conversion" tool for MM that can take current Chem notation and transform it into a queryable-programmable-interactive type model in terms of MM is a good goal. It could get more people on board. In fact I bet a lot of people are curious but since there isn't a clearcut conversion for MM's work to traditional Physics/Chemistry, a lot of people may not take a risk to run with it or investigate it. Building this might also help discover what is complete and what isn't?
MM's algorithms and all the plumbing work to apply them could be useful if it was in a "library" of sorts that could be loaded for doing calcs with MM's numbers. We may need to start Meetups or "sessions" to spread the word to get other people to help do this kind of work....especially younger people.
I keep commenting on using JSON only because it could be flexible enough to allow changes and still allow them to be dropped into frameworks like Nevyn has created. Current database schemas are a bit primitive to represent Mathis' "world" and any revisions. As Mathis fleshes out "bonding" principles between atoms/molecules more and more... these interactions could be programmed in.

Came across this. Would be kind of useful to create even a Mathematica Notebook demonstrating Mathis' theory.

---------
A Mathematica Notebook-CDF example:
http://demonstrations.wolfram.com/QuantumAngularMomentumMatrices/

Mathematica for Physics
Authors
Robert L. Zimmerman
Organization: University of Oregon
Department: Institute of Theoretical Science
Fredrick I. Olness
Organization: Southern Methodist University
Department: Department of Physics
http://library.wolfram.com/infocenter/Books/3588/

Description

Designed as a supplement for any of the core advanced undergraduate and graduate physics courses, Mathematica for Physics chooses the canonical problems from the physics curriculum, and solves these problems using Mathematica. This book takes the reader beyond the "textbook" solutions by challenging the student to cross check the results using the wide variety of Mathematica's analytical, numerical, and graphical tools. Throughout the book, the complexity of both the physics and Mathematica is systematically extended to broaden the tools the reader has at his or her disposal, and to broaden the range of problems that can be solved.

http://demonstrations.wolfram.com/QuantumAngularMomentumMatrices/
http://demonstrations.wolfram.com/topic.html?limit=20&topic=Physics

http://www.wolfram.com/cdf-player/

Book Files:
BookInfo.txt (6.8 KB) - General Information about the book
Download
ReadMe.txt (4.3 KB) - File list, installation instructions and general information
Download
ch1init.m (6.1 KB) - Chapter 1 Initialization
Download
ch1s6p16.nb (8.4 KB) - Section 6 Graphics: Examples 1-6
Download
ch2init.m (7.5 KB) - Chapter 2 Initialization
Download
ch2s2p1.nb (10.8 KB) - Coriolis and Centrifugal Forces
Download
ch2s3p2.nb (9.5 KB) - Uniformly Charged Sphere
Download
ch2s5p2.nb (7.2 KB) - Relativistic Collision
Download
ch3init.m (11.5 KB) - Chapter 3 Initialization
Download
ch3s2p4.nb (8.2 KB) - Motion of a Damped, Forced Nonlinear Pendulum
Download
ch3s3p1.nb (10.1 KB) - Two Coupled Harmonic Oscillators
Download
ch3s3p2a.nb (6.1 KB) - Three Coupled Harmonic Oscillators, Animation Only
Download
ch3s3p3a.nb (6.2 KB) - Double Pendulum, Animation Only
Download
ch4init.m (12.4 KB) - Chapter 4 Initialization
Download
ch4s1p3.nb (10.6 KB) - Bead on a Rotating Hoop
Download
ch4s2p4.nb (8.5 KB) - Numerical Solution for Orbits with Central Forces
Download
ch4s3p3.nb (8.9 KB) - Spherical Pendulum and Hamilton's Equations
Download
ch5init.m (11.5 KB) - Chapter 5 Initialization
Download
ch5s1p1.nb (7.8 KB) - Superposition of Point Charges
Download
ch5s2p5.nb (7.7 KB) - Conducting Cylinder with a Potential on the Surface
Download
ch6init.m (12.4 KB) - Chapter 6 Initialization
Download
ch6s1p2.nb (12.7 KB) - Particle bound in a Finite Potential Well
Download
ch6s1p3a.nb (8.3 KB) - Particle Hitting a Finite Step Potential, Animation Only
Download
ch6s1p4a.nb (7.1 KB) - Particle Propagating Towards a Rectangular Potential, Animation Only
Download
ch6s2p5.nb (8.9 KB) - The Hydrogen Atom in Spherical Coordinates
Download
ch7init.m (15.2 KB) - Chapter 7 Initialization
Download
ch7s1p3.nb (7.3 KB) - Compton Scattering
Download
ch7s2p5.nb (10 KB) - Time It Takes to Fall into a BlackHole

Files specific to Mathematica 2.2 version:
Download
ch1s6p16.ma (7.2 KB) - Section 6 Graphics: Examples 1-6
Download
ch2s2p1.ma (9.2 KB) - Coriolis and Centrifugal Forces
Download
ch2s3p2.ma (8.2 KB) - Uniformly Charged Sphere
Download
ch2s5p2.ma (6.6 KB) - Relativistic Collision
Download
ch3s2p4.ma (7.3 KB) - Motion of a Damped, Forced Nonlinear Pendulum
Download
ch3s3p1.ma (8.4 KB) - Two Coupled Harmonic Oscillators
Download
ch3s3p2a.ma (6.3 KB) - Three Coupled Harmonic Oscillators, Animation Only
Download
ch3s3p3a.ma (6.3 KB) - Double Pendulum, Animation Only
Download
ch4s1p3.ma (8.7 KB) - Bead on a Rotating Hoop
Download
ch4s2p4.ma (7.4 KB) - Numerical Solution for Orbits with Central Forces
Download
ch4s3p3.ma (7.8 KB) - Spherical Pendulum and Hamilton's Equations
Download
ch5s1p1.ma (7 KB) - Superposition of Point Charges
Download
ch5s2p5.ma (6.9 KB) - Conducting Cylinder with a Potential on the Surface
Download
ch6s1p2.ma (10.5 KB) - Particle bound in a Finite Potential Well
Download
ch6s1p3a.ma (7.8 KB) - Particle Hitting a Finite Step Potential, Animation Only
Download
ch6s1p4a.ma (7.1 KB) - Particle Propagating Towards a Rectangular Potential, Animation Only
Download
ch6s2p5.ma (7.9 KB) - The Hydrogen Atom in Spherical Coordinates
Download
ch7s1p3.ma (6.6 KB) - Compton Scattering
Download
ch7s2p5.ma (8.5 KB) - Time It Takes to Fall into a BlackHole

Also came across this recent R "threejs" library. Could be a useful platform for building a Mathis style R object library.  It is static looking but is an interactive dynamic rendering of data points. Click the graphs to pivot them.  

https://bwlewis.github.io/1000_genomes_examples/PCA.html

https://bwlewis.github.io/rthreejs/doughnut/index.html

This could be data-driven from a Mathis style R data library:

https://bwlewis.github.io/rthreejs/pointcloud/index.html

Don't mean to keep harping on this without anything to show for it yet. Looked at this "How to Build an R package" on Cran. 

If the data files contain Mathis' defaults and calcs from his various papers, it could be released and get attention from a pretty wide audience of people?

 It could also be useful in walking people through papers if the Mathis R package is used to help illustrate the detailed maths involved. Most of this would be a library of his functions.

https://cran.r-project.org/doc/contrib/Leisch-CreatingPackages.pdf
http://cran.r-project.org/doc/manuals/R-exts.pdf
http://www1.appstate.edu/~arnholta/Software/MakingPackagesUnderWindows.pdf

https://r-forge.r-project.org/

A place to submit and build an R program.

Cr6, To be perfectly clear, I'm drawing a blank. I am the least qualified among us to judge the relative merits of various applications packages designed for information presentation within contemporary Library Science.

I don't understand what you're asking for.

How may I help you? Tell me what to do.
.

Sorry LTAM...

Here's the step by step (for Windows):

1. Download RBase:
https://cran.r-project.org/bin/windows/base/

2. Download RStudio:
https://download1.rstudio.org/RStudio-0.99.903.exe

3. Open RStudio and click Help > Update
select a CRAN location and get the latest updates (R often requires package updating to get the latest features).

4. start with a simple intro site for simple math and assigning variables
https://cran.r-project.org/doc/contrib/Lemon-kickstart/index.html
http://tryr.codeschool.com/
http://trevorstephens.com/kaggle-titanic-tutorial/getting-started-with-r/
https://www.kaggle.com/c/titanic/details/new-getting-started-with-r
https://www.r-project.org/doc/bib/R-books.html


5. Install additional libraries as needed.
Fire up R and create an .r file.
Paste something like this in the window and run. (# is a comment).

 # Now we download, install and initialize the H2O package for R.
 install.packages("h2o", type="source", repos=(c("http://h2o-release.s3.amazonaws.com/h2o/rel-simons/4/R")))
 install.packages("deepnet")
 install.packages("rvest")
 install.packages("XML")
 install.packages("data.table")
 install.packages("clusterGeneration")

6. Work towards these scripts:
http://www.gcdataconcepts.com/coaster_script.html
http://www.gcdataconcepts.com/coaster_analysis.r
http://trevorstephens.com/competitions/armchair-particle-physicist/
https://github.com/tqchen/xgboost/tree/master/demo/kaggle-higgs
https://github.com/hetong007/higgsml
https://github.com/wehrley/wehrley.github.io/blob/master/SOUPTONUTS.md

7. Work towards building Mathis library with the Excel and Javascript files dumped to data files that can be loaded into R. Attempt to build functions that match Mathis' papers using his transforms-corrections, etc.
https://support.rstudio.com/hc/en-us/articles/200486488-Developing-Packages-with-RStudio
http://www.stt.msu.edu/~cui/Groupmeeting/R_package_tutorial.pdf

.
OK, I've got the RGui.



rsesssion.exe is giving me system errors because rgraphapp.dll and r.dll are missing.

It's mostly foreign looking, I'll read up on it.

I just may send personal correspondence for those embarassing moments.

.

We need a little test project to get a feel for what R can do for us. I don't know much about R but it seems to be a math visualisation package so we just need some math we want to visualise. You could take the math we discussed in the How to determine gravity thread, implement it in R and figure out how to show it.

Another potential project is to take Miles' angular velocity equation and plugin a doubling radius to show the angular velocity changes as new spin levels are added to a BPhoton. Then analyze the data from that to show the relationship between spin levels (take the ratio of each spin level to the level on top of it). That is how I found the spin level relationships that SpinSim is now based on.

Something I have been thinking about for a few years is to visualise the energy of photons as spins are added. Relate it back to the standard EM energy chart. It would be good to have a quick chart to see how stacked spins relate to photon frequency and wavelength, etc.

LongtimeAirman wrote:.
OK, I've got the RGui.


rsesssion.exe is giving me system errors because rgraphapp.dll and r.dll are missing.

It's mostly foreign looking, I'll read up on it.

I just may send personal correspondence for those embarassing moments.

.

Just wanted to check that you are 64bit to 64bit across R.exe, RStudio and OS? Also you may need to set write permissions to the installation directory of R base.

You might want to look at starting with RStudio since it is easier to get started with than Rgui.  Just Open RStudio Menu > Tools > Options > set R install location (64 bit OS with 64 bit Rstudio and 64 bit R base install).
If this throws the .dll errors, then there should be more description of the issue from RStudio.

Two packages-libraries you should look at installing are the R Excel libraries to load Excel files.
install.packages("xlsx")
install.packages("XLConnect")
https://www.datacamp.com/community/tutorials/r-tutorial-read-excel-into-r

That's exactly the idea Nevyn.  Bring Mathis' and (yours) calcs-functions into an R library to support  findings.  Many R packages have a small file that is a sample data frame ( embedded .csv file) -- it would be cool to drop in a list of default values similar to the NIST values.

.
OK, I've got RStudio. I was incorrectly trying to run RSession instead (D'Oh).

I've been through the RManual chap1 and Appendix A with follow along excersizes. That went well. Very nice plotting possibilities indeed.

More later.
.

Cr6, Nevyn,

I’ve used excel often. Mainly to create various charts. Keeping a small set of required forms with embedded calcs and links current over many years was part of my last job. Aside from that, calculators, and dimly recalled card deck FORTRAN, computational s/w is new to me. I'll admit it, R does seem very interesting, but I still can’t imagine Data Analysis.

R Update:
I’ve been through the O’Reilly’s tutorial - http://tryr.codeschool.com/
I’ve believe I’ve downloaded the packages (copy/pasted from my .Rhistory):
install.packages("h2o", type="source", repos=(c("http://h2o release.s3.amazonaws.com/h2o/rel-simons/4/R")))
install.packages("deepnet")
install.packages("rvest")
install.packages("XML")
install.packages("data.table")
install.packages("clusterGeneration")
install.packages("xlsx")
install.packages("XLConnect")

Currently at: https://www.datacamp.com/ R Tutorial. Reading and Importing Excel Files into R. I ‘cleaned up’ (by simplifying as much as possible) an excel copy of http://nssdc.gsfc.nasa.gov/planetary/factsheet/index.html PlanetaryFactSheetMetric.txt (preliminary to How to determine gravity thread doc). No joy porting it into R yet.

Interesting text: Empirical Research in Economics, Growing up with R
http://csun.cfr.msstate.edu/docs/ererSamplePage211.pdf

I didn't get very far, it was a busy week. I’ll continue to follow both your directions.
.

LTAM and Nevyn,

Sounds like you guys are off to a good start.  There are developer tools for R that allows for the creation of custom libraries/functions. There are a lot of R libraries and it is easy to get swamped by them. I'll try and keep an eye on those that might be useful to help create a "Mathis" library. If I get something together that appears to work, I'll share it here.

If this becomes too burdensome... (or just time-wasting)... don't let it detract from other work/insights.  It takes awhile to get going with it. A lot of the imported text files have to "munged" with datatypes (e.g. numeric conversions) which is very time consuming.

I'll see what head-way I can make from Nevyn's suggested Gravity thread.

R Update. So far so good. I continued with a nice BestFirstRTutorial.pdf. Turns out RStudio makes porting tables pretty easy with a walk-you-through step-by-step wizard.  

Trying to tailor a first project - How to determine Gravity?

What I currently imagine is starting with the following outline, then expanding it by loading all the appropriate equations.

Code:

##----------------------------------------------------
##
## What are the Planets' gravity and charge emission?
##
##----------------------------------------------------
## A Miles Mathis charge field R Script.
##    first project effort
##----------------------------------------------------
## Miles Mathis reference, The Moon Gives up a Secret
##      http://milesmathis.com/moon.html
##----------------------------------------------------
## The planetary source data is from NASA:
##    http://nssdc.gsfc.nasa.gov/planetary/factsheet/index.html
##    Modified as an excel doc and saved as
##    Planetary Fact Sheet Metric.txt
##----------------------------------------------------
## Some discussion. How to determine Gravity  
##    http://milesmathis.forumotion.com/t205-how-to-determine-gravity
##----------------------------------------------------
## Generally, everything here is in meters, seconds and
##    kilograms; density is kg/m^3, velocity is m/s,
##    acceleration is m/s^2, and mass is kg.
##----------------------------------------------------
## Let:
##    r = radius, (or r = diameter/2);
##    g = solo gravity;
##    d = density;
##    m = mass;
##    e = electromagnetic (E/M) emission;
##    v = apparent gravity.
##----------------------------------------------------
##  First example.
##    v_E = g_E + e_E = 9.80 (m/s^2)
##    v underscore E refers to earth's
##    apparent gravity. Apparent gravity is equal to solo
##    gravity plus E/M emission. The goal is to understand  
##    the given relationship. Is it possible to make this
##    equation interactive?
##----------------------------------------------------
##  In The Moon Gives up a Secret, Miles expresses
##    the Moon's: mass, radius and gravity; with
##    respect to Earth's.
##  Compared to Earth, the Moon's:
##    mass, m_M/m_E = 1/81;
##    radius, r_M/r_E = 1/3.67;
##    and gravity at the surface, v_M/v_E = 1/6.
##  Miles determines:
##    Earth's E/M emission, e_E = .009545 (m/s^2)
##    Moon's E/M emission, e_M = 1.051 (m/s^2)
##    Earth's solo gravity, g_E = 9.809545 (m/s^2)
##    Moon's solo gravity, g_M = 2.671 (m/s^2)
##----------------------------------------------------
##  Second example.
##    Work through the logic to obtain these results
##----------------------------------------------------
##  Third example.
##    What are the solo gravities and charge emissions
##    of the planets?

Does this sound realistic/reasonable? I don't know how interactive R can be, and I have no best practices to compare to yet.
.

That sounds like a great plan LTAM. I've actually been working too much with this library lately:

https://cran.rstudio.com/web/packages/FFTrees/vignettes/FFTrees_overview.html

I've managed to find a way to get paid to look into R!

I'm about to take a month off work and this is my last week so my workload is a little light. A colleague was asking me about some system monitoring software we might be able to use and when he stated what he wanted to do, I suggested that I could look into R if he thought it would fit his requirements, which it did. So I get to play around with it during work hours and we get the knowledge for our projects here. That's the way life should work!

I think I will use Miles angular velocity equation as a test function and see what I can get out of it.

Just to be clear, R is not a system monitoring system but we will be using it to analyze the results as we do some load testing on some software we have built.

I haven't gotten very far into R yet, but from what I have seen, it is a very ugly language. Powerful, but ugly. Very Unix like with lots of abbreviated commands that you won't remember. I'm also struggling a bit because I don't understand a lot of the math they are using for examples. It will probably start making more sense when I begin writing my own equations and have some idea of what I want to do with them.

Nevyn wrote:I haven't gotten very far into R yet, but from what I have seen, it is a very ugly language. Powerful, but ugly. Very Unix like with lots of abbreviated commands that you won't remember. I'm also struggling a bit because I don't understand a lot of the math they are using for examples. It will probably start making more sense when I begin writing my own equations and have some idea of what I want to do with them.

Yeah... it is like that. Like there are 6 different ways to do the same thing each a little cryptic and different from the other...kind of like PERL in a way. It mixes set and procedural concepts which can often be a problem.  There are a lot of  "artistic" or "Swedish Chef" ala "recipe" approaches over say an architectural approach or an approach with blue prints.  It is still one of the only languages where a lot of code has been converted from cpp and pascal for use.  

That's awesome that you have time to really look into the R language.  It can be rewarding and it does work with "Big Data" on the cheap unlike a lot of other languages.  It does have past and a pretty bright future. That's why I'm kind of leaning towards creating a lot of Mathis' equations/theorems with it in a published package. It can get attention from a lot of people.
It is becoming the "Truth Bench" for a lot people.  

.
An update, proof that I'm still at it.  


y <- (0.3 * sin(25*x)) + (sin(x))
.

LongtimeAirman wrote:.
An update, proof that I'm still at it.  


y <- (0.3 * sin(25*x)) + (sin(x))
.

Looking good LTAM.. Sorry I've been really lazy lately in getting something cranked out.

Hey Guys,

I created a Periodic Table library for R.  The package is "PeriodicTable" and needs to be installed manually. Note it requires R 3.3.2 to compile.

It needs to be installed manually from RStudio instead of from CRAN. I haven't added an special functions or calcs to it. It is basically a dump of the Excel files I had posted earlier.

http://www.filedropper.com/periodictable001rx8664-pc-linux-gnutar

What does it do, Cr6?

I updated to R version 3.3.2 (2016-10-31) -- "Sincere Pumpkin Patch".
From within RStudio I see:
1, A three line file INDEX) PeriodicTable    titanic: Periodic Table and Miles Mathis' data sets (Feed the webkitty! www.milesmathis.com)
2, A twenty line file DESCRIPTION) Package, Title, Version, ..., Build
3, A twenty seven line file PeriodicTable) #  File share/R/nspackloader.R
Nothing else opens for me.
Also, I've tried reloading. The zip extractor gives this message:

////////////////////////////////////////////////////////
No Joy loading from the RStudio console.
install.packages("C:/Users/Robert/Downloads/PeriodicTable_0.0.1_R_x86_64-pc-linux-gnu.tar.gz", repos = NULL, type = "source")
Installing package into ‘C:/Users/Robert/Documents/R/win-library/3.3’ (as ‘lib’ is unspecified)
Error in untar2(tarfile, files, list, exdir, restore_times) :
 incomplete block on file Warning in install.packages :
 running command '"C:/PROGRA~1/R/R-33~1.2/bin/x64/R" CMD INSTALL -l "C:\Users\Robert\Documents\R\win-library\3.3" "C:/Users/Robert/Downloads/PeriodicTable_0.0.1_R_x86_64-pc-linux-gnu.tar.gz"' had status 1
Warning in install.packages :
 installation of package ‘C:/Users/Robert/Downloads/PeriodicTable_0.0.1_R_x86_64-pc-linux-gnu.tar.gz’ had non-zero exit status

Yeah.... I haven't tried it on Windows yet. You might try and install it directly via R Studio. Install packages -> local packages -> the .gz file.

I might need to recompile it on Windows with Rtools for it to work there. I might need to correct the "titanic" errors since I was using it as a template. I can recompile it as a .zip file which may work better on Windows.

Essentially the purpose of the file is to have an "R" library like the library(titanic) which has a file of details.  This is a file on the Periodic table and can be used for datamining/calcs/functions.  Three data tables are included NIST, MathisCF (Charge Field), and PeriodicTable.  These are dumps of the Excels I cobbled from various sources (mostly Wikipedia). It's not too fancy but can be used as a reference point. Basically, it can be included in R scripts and the tables used for custom functions. Was using this as a trial run on building an "R" package from scratch.

Here's the R script.
Excel file:
http://www.filedropper.com/mathisallelementsv30

Place in a folder and change the datDir location below:

Code:


library(xlsx)
library(XLConnect)
datDir <- "/usr/local"
files <- list.files(datDir, pattern = ".xlsx",full.names=T)
#files
#file <- system.file("mathis", files, package = "xlsx")
MathisCF <- read.xlsx(files, 1, colIndex = 1:12)  # read first sheet

NIST <- read.xlsx(files, 2)  # read first sheet
PeriodicTable <- read.xlsx(files, 3)  # read first sheet
PeriodicTable[is.na(PeriodicTable)] <- 0
MathisCF[is.na(MathisCF)] <- 0
library(qdap)
PeriodicTable <- replacer(PeriodicTable, NA, "")
MathisCF <- replacer(MathisCF, NA, "")

MathisCF
PeriodicTable
NIST

# rm(datDir, files)
# rm(list=ls())
# ls()
# getwd()
#
# setwd("/home/defaultuser/PeriodicTable")
# library(devtools)
# create("PeriodicTable")
# use_data(MathisCF,PeriodicTable,NIST, pkg = "PeriodicTable", internal = FALSE, overwrite = FALSE,
#          compress = "bzip2")
#
# install("PeriodicTable")
#
# devtools::use_data()
# x <- 3
# y <- list(a=TRUE,b="good")
# save(PeriodicTable,file="PeriodicTable.RData")
# save(NIST,file="NIST.RData")
# save.image("Mathis.RData")
#
# setwd("/home/defaultuser/")
# load("Mathis.RData")

library(PeriodicTable)
head(PeriodicTable)

library(sparklyr)
library(rsparkling)
library(h2o)
library(dplyr)
periodic_tbl <- copy_to(sc, PeriodicTable , "PeriodicTable", overwrite = TRUE)



.
Cr6, Please excuse the delay. We're still in recovery, having hosted a large family and friends Turkeyday Feast here. I'm lucky to need just a new stove.

I can copy individual tabs to import into R, but prefer to do things correctly, as well as using the data. Didn't you say it might take some time? Thanks for v30!
.

.
My third and final educational purchase for the year is The Book of R by Tilman M. Davies, A First Course in Programming and Statistics. It seems very comprehensive and I'm getting more motivated each time I look at it.  
.

LongtimeAirman wrote:.
My third and final educational purchase for the year is The Book of R by Tilman M. Davies, A First Course in Programming and Statistics. It seems very comprehensive and I'm getting more motivated each time I look at it.  
.

Sorry LTAM.... been watching too much College Football lately...and generally goofing around with non-Mathis/non-important stuff....  (I'm sure you understand.).  

That's a good "foundation" book from the reviews I've seen.  I think my package may have had issues with includes of required libraries like "qdap". I may not waste too much time to perfect it and just leave the script to load the .xlsx file.  Keeping the data types consistent is always a challenge though when loading from Excel.  The coolest thing about R scripts is that they can be loaded up in a cloud server to run over a massive database.  This is one cool thing that it provides. If ever at some point a SMILES or InChI dataset could be translated to a Mathis-3D molecule based on Mathis' famework... it would be heaven.  http://www.rdkit.org/docs/Cartridge.html   http://tech.knime.org/book/chemical-identifier-resolver-for-knime-trusted-extension (chemical identifier resolver).

Dragon 7.0
https://chm.kode-solutions.net/products_dragon.php (commercial version).
https://chm.kode-solutions.net/products_dragon_descriptors.php

Just an update on this. Cran has a new PeriodicTable package from a Frenchman.


https://cran.rstudio.com/web/packages/PeriodicTable/
https://cran.rstudio.com/web/packages/PeriodicTable/PeriodicTable.pdf

PeriodicTable: Periodic Table of the Elements

Provides a dataset containing properties for chemical elements. Helper functions are also provided to access some atomic properties.
Version: 0.1.1
Depends: R (≥ 3.3.1)
Published: 2017-01-24
Author: Julien Idé
Maintainer: Julien Idé

Project Malmo – a platform for fundamental AI research

(Vid at around 20 minutes has a sample of )
https://www.youtube.com/watch?v=399qJUBRA0o
https://github.com/Microsoft/CNTK/wiki/Object-Detection-using-Fast-R-CNN#train-on-your-own-data

Newbie intro:
https://www.youtube.com/watch?v=2FOXR16mLow
Newbie build Game AI:
https://ww.youtube.com/watch?v=HBAUeJkFMH0

CNTK:

https://www.youtube.com/watch?v=AJwn7doMiU0
https://www.microsoft.com/en-us/research/product/cognitive-toolkit/

DeepMind (Google)

https://deepmind.com/blog/open-sourcing-deepmind-lab/

DeepMind Lab is a fully 3D game-like platform tailored for agent-based AI research. It is observed from a first-person viewpoint, through the eyes of the simulated agent. Scenes are rendered with rich science fiction-style visuals. The available actions allow agents to look around and move in 3D. The agent’s “body” is a floating orb. It levitates and moves by activating thrusters opposite its desired direction of movement, and it has a camera that moves around the main sphere as a ball-in-socket joint tracking the rotational look actions. Example tasks include collecting fruit, navigating in mazes, traversing dangerous passages while avoiding falling off cliffs, bouncing through space using launch pads to move between platforms, playing laser tag, and quickly learning and remembering random procedurally generated environments. An illustration of how agents in DeepMind Lab perceive and interact with the world can be seen below:



https://deepmind.com/blog/reinforcement-learning-unsupervised-auxiliary-tasks/

Pretty interesting package I came across recently:

https://beckmw.wordpress.com/2017/04/12/predicting-tides-in-r/#tldr


Predicting tides in R

Skip to TL/DR….

Water movement in estuaries is affected by many processes acting across space and time. Tidal exchange with the ocean is an important hydrodynamic process that can define several characteristics of an estuary. Physical flushing rates and water circulation are often controlled by tidal advection, whereas chemical and biological components are affected by the flux of dissolved or particulate components with changes in the tide. As such, describing patterns of tidal variation is a common objective of coastal researchers and environmental managers.

Tidal predictions are nothing new. A clever analog approach has been around since the late 1800s. The tide-predicting machine represents the tide as the summation of waves with different periods and amplitudes. Think of a continuous line plot where the repeating pattern is linked to a rotating circle, Representing the line in two-dimensions from the rotating circle creates a sine wave with the amplitude equal to the radius of the circle. A more complex plot can be created by adding the output of two or more rotating disks, where each disk varies in radius and rate of rotation. The tide-predicting machine is nothing more than a set of rotating disks linked to a single graph as the sum of the rotations from all disks. Here’s a fantastic digital representation of the tide-predicting machine:




Tides are caused primarily by the gravitational pull of the sun and moon on the earth’s surface. The elliptical orbits of both the moon around the earth and the earth around the sun produce periodic but unequal forces that influence water movement. These forces combined with local surface topography and large-scale circulation patterns from uneven heating of the earth’s surface lead to the variation of tidal patterns across the globe. Although complex, these periodic patterns can be characterized as the summation of sine waves, where one wave represents the effect of a single physical process (e.g., diurnal pull of the moon). Describing these forces was the objective of the earlier tide-predicting machines. Fortunately for us, modern software (i.e., R) provides us with a simpler and less expensive approach based on harmonic regression.

Approach

We’ll create and sum our own sine waves to demonstrate complexity from addition. All sine waves follow the general form y as a function of x:



where the amplitude of the wave is beta and the frequency (or 1 / period) is f. The parameters alpha and Phi represent scalar shifts in the curve up/down and left/right, respectively. We can easily create a function in R to simulate sine waves with different characteristics. This function takes the parameters from the above equation as arguments and returns a sine wave (y) equal in length to the input time series (x). The alpha and beta are interpreted as units of wave height (e.g., meters) and f and Phi are in hours.

# function for creating sine wave
waves <- function(time_in, alpha = 0, beta = 1, freq = 24, phi = 0){

# timestep per hour
time_step <- 60 / unique(diff(time_in))

# set phi as difference in hours from start of time_in
phi <- min(time_in) + phi * 3600
phi<- as.numeric(difftime(phi, min(time_in)))
phi <- phi / time_step

# get input values to cos func
in_vals <- seq(0, length(time_in), length = length(time_in))
in_vals <- in_vals / time_step
in_vals <- 2 * pi * in_vals * 1 / freq

# wave
y <- alpha + beta * sin(in_vals + phi)

return(y)

}

The default arguments will return a sine wave with an amplitude of one meter and frequency of one wave per 24 hours. Two additional time series are created that vary these two parameters.

# input time series for two weeks, 15 minute time step
x <- as.POSIXct(c('2017-04-01', '2017-04-15'))
x <- seq(x[1], x[2], by = 60 * 15)

# get three sine waves
# a: default
# b: amplitude 0.5, 48 hour period
# c: amplitude 2, 12 hour period
a <- waves(x)
b <- waves(x, beta = 0.5, f = 48)
c <- waves(x, beta = 2, f = 12)

We can combine all three waves in the same data object, take the summation, and plot to see how it looks.

# for data munging and plotting
library(tidyverse)

# get sum of all y values, combine to single object
yall <- rowSums(cbind(a, b, c))
dat <- data.frame(x, a, b, c, yall) %>%
gather('var', 'val', -x)

# plot
ggplot(dat, aes(x = x, y = val)) +
geom_line() +
facet_wrap(~var, ncol = 1) +
theme_bw()



The important piece of information we get from the plot is that adding simple sine waves can create complex patterns. As a general rule, about 83% of the variation in tides is created by seven different harmonic components that, when combined, lead to the complex patterns we observe from monitoring data. These components are described as being of lunar or solar origin and relative periods occurring either once or twice daily. For example, the so-called ‘M2’ component is typically the dominant tidal wave caused by the moon, twice daily. The periods of tidal components are constant across locations but the relative strength (amplitudes) vary considerably.

maincomponents

The oce package in R has a nifty function for predicting up to 69 different tidal constituents. You’ll typically only care about the main components above but it’s useful to appreciate the variety of components included in a tidal signal. We’ll apply the tidem function from oce to predict the tidal components on a subset of SWMP data. A two-week period from the Apalachicola Bay Dry Bar station is used.

library(SWMPr)
library(oce)

# clean, one hour time step, subset, fill gaps
dat <- qaqc(apadbwq) %>%
setstep(timestep = 60) %>%
subset(subset = c('2013-01-01 0:0', '2013-12-31 0:0'), select = 'depth') %>%
na.approx(maxgap = 1e6)

The tidem function from oce requires a ‘sealevel’ object as input. Plotting the sealevel object using the plot method from oce shows three panels; the first is the complete time series, second is the first month in the record, and third is a spectral decomposition of the tidal components as cycles per hour (cph, or period).

datsl <- as.sealevel(elevation = dat$depth, time = dat$datetimestamp)
plot(datsl)

We can create a model to estimate the components from the table above using tidem. Here, we estimate each component separately to extract predictions for each, which we then sum to estimate the complete time series.

# tidal components to estimate
constituents <- c('M2', 'S2', 'N2', 'K2', 'K1', 'O1', 'P1')

# loop through tidal components, predict each with tidem
preds <- sapply(constituents, function(x){

mod <- tidem(t = datsl, constituent = x)
pred <- predict(mod)
pred - mean(pred)

})

# combine prediction, sum, add time data
predall <- rowSums(preds) + mean(datsl[['elevation']])
preds <- data.frame(time = datsl[['time']], preds, Estimated = predall)

head(preds)

## time M2 S2 N2 K2
## 1 2013-01-01 00:00:00 -0.111578526 -0.020833606 0.000215982 -0.0048417234
## 2 2013-01-01 01:00:00 -0.118544835 -0.008940681 0.006428260 -0.0093752262
## 3 2013-01-01 02:00:00 -0.095806627 0.005348532 0.011088593 -0.0113830570
## 4 2013-01-01 03:00:00 -0.049059634 0.018205248 0.013072149 -0.0103243372
## 5 2013-01-01 04:00:00 0.009986414 0.026184523 0.011900172 -0.0064842694
## 6 2013-01-01 05:00:00 0.066540974 0.027148314 0.007855534 -0.0008973087
## K1 O1 P1 Estimated
## 1 0.0911501572 0.01312209 0.0381700294 1.463683
## 2 0.0646689921 0.03909021 0.0340807303 1.465686
## 3 0.0337560517 0.06274939 0.0276811946 1.491713
## 4 0.0005294868 0.08270543 0.0194051690 1.532812
## 5 -0.0327340223 0.09778235 0.0098135843 1.574727
## 6 -0.0637552642 0.10709170 -0.0004434629 1.601819

Plotting two weeks from the estimated data shows the results. Note the variation in amplitude between the components. The M2 , K1, and O1 components are the largest at this location. Also note the clear spring/neap variation in range every two weeks for the combined time series. This complex fort-nightly variation is caused simply by adding the separate sine waves.

# prep for plot
toplo <- preds %>%
gather('component', 'estimate', -time) %>%
mutate(component = factor(component, level = c('Estimated', constituents)))

# plot two weeks
ggplot(toplo, aes(x = time, y = estimate, group = component)) +
geom_line() +
scale_x_datetime(limits = as.POSIXct(c('2013-07-01', '2013-07-31'))) +
facet_wrap(~component, ncol = 1, scales = 'free_y') +
theme_bw()

All tidal components can of course be estimated together. By default, the tidem function estimates all 69 tidal components. Looking at our components of interest shows the same estimated amplitudes in the plot above.

# estimate all components together
mod <- tidem(t = datsl)

# get components of interest
amps <- data.frame(mod@data[c('name', 'amplitude')]) %>%
filter(name %in% constituents) %>%
arrange(amplitude)
amps

## name amplitude
## 1 K2 0.01091190
## 2 N2 0.01342395
## 3 S2 0.02904518
## 4 P1 0.04100388
## 5 O1 0.11142455
## 6 M2 0.12005114
## 7 K1 0.12865764

And of course comparing the model predictions with the observed data is always a good idea.

# add predictions to observed data
dat$Estimated <- predict(mod)

# plot one month
ggplot(dat, aes(x = datetimestamp, y = depth)) +
geom_point() +
geom_line(aes(y = Estimated), colour = 'blue') +
scale_x_datetime(limits = as.POSIXct(c('2013-07-01', '2013-07-31'))) +
scale_y_continuous(limits = c(0.9, 2)) +
theme_bw()

The fit is not perfect but this could be from several reasons, none of which are directly related to the method – instrument drift, fouling, water movement from non-tidal sources, etc. The real value of the model is we can use it to fill missing observations in tidal time series or to predict future observations. We also get reasonable estimates of the main tidal components, i.e., which physical forces are really driving the tide and how large are the contributions. For example, our data from Apalachicola Bay showed that the tide is driven primarily by the M2, K1, and O1 components, where each had relative amplitudes of about 0.1 meter. This is consistent with general patterns of micro-tidal systems in the Gulf of Mexico. Comparing tidal components in other geographic locations would produce very different results, both in the estimated amplitudes and the dominant components.
TL/DR

Here’s how to estimate the tide from an observed time series. The data are from SWMPr and the tidem model is from oce.

library(SWMPr)
library(oce)

# clean input data, one hour time step, subset, fill gaps
dat <- qaqc(apadbwq) %>%
setstep(timestep = 60) %>%
subset(., subset = c('2013-01-01 0:0', '2013-12-31 0:0'), select = 'depth') %>%
na.approx(maxgap = 1e6)

# get model
datsl <- as.sealevel(elevation = dat$depth, time = dat$datetimestamp)
mod <- tidem(t = datsl)

# add predictions to observed data
dat$Estimated <- predict(mod)

# plot
ggplot(dat, aes(x = datetimestamp, y = Estimated)) +
geom_line() +
theme_bw()